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Introduction 

In the 1984-85 school year, the Pennsylvania Department of Education 
(PDE) , at the governor's behest, began to actively encourage Pennsylvania 
school districts to reform their teacher supervision/evaluation (TS/E) 
procedures. Its goal was to Improve the quality of classroom Instruction 
and make teacher evaluation more consistent and meaningful* As an Initial 
step PDE, In cooperation with the state Intermediate Units, sponsored a 
series of 29 regional two-day workshops to familiarize school staff with the 
components of effective TS/E systems and the Btaff skills required to 
Implement them (McGreal^ 1983)* In 1985-86, PDE plans to provide districts 
with more specific models and how-to-do-lt information based on existing 
TS/E systems In the state. This will be followed by technical assistance 
In the developmiint and Implementation of similar TS/E systems* 

To obtain some of the data necessary to this effort, PDE commissioned 
RBS to design and conduct this study of five school district TS/E systems. 
The five districts were selected by PDE after a questionnaire and phone 
Survey determined which districts In the state had TS/E systems either In 
place or evolving. An attempt was also made to Include urban, suburban, 
and rural sites, among the districts selected. In the end, Ablngton 
Heights, Pittsburgh, and Upper Perklomen were selected because they had 
Madeline Hunter-type Instructional Improvement /supervisory models In use. 
Suburban East Penn and rural Tamaqua were selected for their evolving 
alternative TS/E systems. 
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Method 

The study took a case gtudy approach. Separate interview protocols 
for district central office staff, administrators/trainers and teachrrs 
were developed and pilot tested. These were based in part on the protocols 
used in a recent Rand Corporation study of effective teacher evaluation 
practices (Wise, et al., 1984, personal communication) • The interviews 
also addressed some 25 how-to-do-it type questions posed over the past year 
by Pennsylvania school officials in TS/E related meetings and conversations. 

In addition, the terms "supervision" and "evaluation" were defined at 
the beginning of each interview to ensure a common understanding among 
participants and researchers. Supervision was defined as that cycle of 
activities between a teacher and an administrator or supervisor that is 
intended to improve the teacher's ability to perfoinn in the class. It is 
primarily improvement oriented and can focus on instructional techniques, 
class management, planning, implementation of the district curriculum, and 
so forth. It can focus on teacher improvement goals and/or district goals. 
Evaluation was defined as the culmination of the supervision cycle wherein 
the administrator or supervisor makes a summary judgment or evaluation of 
the teacher's classroom performance for personnel or accountability 
purposes — usually on aii annual basis.* 

A three-phese study procedure was devised. First, descriptive TS/E 
system background and policy information (e.g., policy manuals, training 



*In Pennsylvania, the annual evaluation of teachers and other profes- 
sional employees is mandated (c.f. Pennsylvania Education code, Chapter 
351, 351.21 Rating Form). The state form for this purpose, the Temporary 
Professional Employee/Professional Employee Rating Form, is referred to 
here as DEBE-333. 
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materials, observation Instruments, budget data) was solicited from each of 
the five districts* ■ A brief pre slt^a-vlslt background Information question- 
naire was also used* Second, In the spring of 1985, two researchers spent 
three days on site at each district (six person days per site) Interviewing 
school staff* At each site they Interviewed central office staff and dis- 
trict trainers for approximately three hours, administrators for one-and- 
one-half hours, and teachers (separately or In small groups) for a half-hour. 
In the four smaller districts, Tamaqua, Upper Perklomen, Ablngton Heights, 
and East Penn> the number of administrators (Including central office staff) 
and teachers Interviewed ranged from 7 to 10 and 31 to 43, respectively. In 
Pittsburgh, the largest district » 22 administrative staff and 51 teachers 
were interviewed either separately or In small groups* Across the five 
districts, a total of 20 central office staff, 37 administrators and/or 
trainers, and 194 teachers were interviewed* 

Interviews with central office staff touched upon development processes, 
system designs and operations, and outcomes, strengths and weaknesses* 
Interviews with administrators focused primarily on their specific roles in 
supervision and evaluation, the utility of the training they received, their 
perceptions of their Impact, perceived system strengths and weaknesses, and 
recommended changes* Interviews with teachers focused primarily on their 
perceptions of the utility. Impact and fairness of the training/supervision 
and evaluation procedures, the specifics of the procedures, perceived 
strengths and weaknesses, and recommended changes. Several questions were 
asked of all staff to determine if they shared common perceptions of the 
goals, procedures and Impact of the TS/E system^ 
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Lastly, follow-up contacts with select district officials were made for 
fxirther clarification of the information collected. In essence, district 
staff verified the accuracy of the program descriptions and implementation 
procedures cited in the report «, 
Related Research 

Two recent publications address factors related to effective teacher 
supervision/evaluation systems, McGreal (1983) identified nine factors 
shared by effective teacher evaluation systems; (1) focus on instructional 
improvement as opposed to accountability; (2) correspondence between the 
major purpose (s) of supervision/evaluation and procedures and instrumenta- 
tion; (3) separation of evaluation of teaching (supervision for improvement 
purposes) from teacher evaluation; (4) use of some form of goal setting 
procedures (individualization of supervisory procedures); (5/ a narrow focus 
on teaching and a common understanding among administrators and teachers of 
the teaching act; (6) use of a modified clinical supervision format; (7) use 
of alternative sources of data; (8) different requirements for tenured and 
non-tenured teachers; and (9) in-depth training or staff development for 
both administrators and teachers. 

The second publication was a recent Rand study (Wlsei 1984) of teacher 
evaluation systems in 32 school districts. Four of the districts were 
studied intensively. Researchers found well-developed supervision and 
evaluation systems in only a few of the districts. The problems most 
frequently cited in the districts were: (I) lack of sufficient resolve and 
competence on the part of principals to evaluate accurately; (2) teacher 
resistance to and lack of support for district evaluation programs because 
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of apathy and perceptions of inconsistent criteria and subjective vari-- 
ability in the evaluation process; and (3) lack of credibility in cases 
where generalists (principals) evaluated specialists (high school and/or 
special subject area teachers) « The most commonly cited benefits^ even in 
less well-developed teacher evaluation systems, were: (1) improved adminis- 
trator-teacher communications; and (2) increased teacher awareness of 
instructional goals and classroom practices, due in part to the use of 
modified clinical supervision procedures* 

The characteristics which distinguished the more successful from the 
less successful districts were: (1) provision of top-level leadership and 
resources for the evaluation program; (2) provision of training for evalua- 
tors in the skills required to evaluate effectively; (3) teacher-evaluator 
goals and procedures; and (4) development and implementation of evaluation 
procedures and support systems that are integrated with the district's 
overall goals and organizational structure. 
The Report 

This report presents the detailed case study of each of the five 
districts^ A related document contains a summary of the case studies, 
and a detailed discussion of the study findings and recommendations* 
Study Findings 

Analyses of the data generally indicate that the study results provide 
support for many of the characteristics of effective TS/E systems identified 
by the Rand study and McGreal's publication* 
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CASE STUDY 1 



Ablngton Heights School District 
218 East Grove Street 
Clarks Summit, PA 1 84 11 
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CASE STUDY 1: ABINGTON HEIGHTS SCHOOL DISTRICT 



Background DJta 



Schools 



Students 



4 Elementary (K-4) 

1 Middle (5-8) 

2 High Schools 
(9-10) & (11-12) 



• 3,364 students 

• 99% white 

• Student population 



declining 



Staff 



• Superintendent 

• Assistant Superintendent for Personnel and Instructional Services 

• Business Manager 

• Director of Public Relations 

• 3 Elementary School Principals 

• 1 Middle School Principal and an Assistant Principal 

• 1 High School Principal and 2 Assistant Principals 

• 8 Subject Area Coordinators/Supervisors (supervisory certificated) 

• 5 Department Heads (non-supervisory) 

• 99 Elementary School Teachers 

• 119 Secondary School Teachers 

(74% of staff « 10+ years of experience) 

Average Per-Pupil Expenditure 

• $3,381 

Teacher Supervision/Evaluatio n 

• Modified DEBE-333 procedure, instituted in 1975-1976, 

• Hunter -based staff development initiative, Program for More Effective 
Teaching (PMKT) initiated in 1979. 

• Teacher evaluation indirectly referenced in collective bargaining 
agreement with PSEA affiliated teachers' association* 



The Abington Heights School District which is located about eight 
miles north of Scranton, Pennsylvania is comprised of eight growing 
boroughs and townships populated by approximately 21,000 residents* The 
boroughs and townships may be classified as basically middle and upper 
middle class professional communities* Relatively little industry exists 
in the district, and the one rural area in the district is diminishing in 
size as more land is being divided into building sites* 



Communitv 
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Program Initiation, Flannlngt and Start"*up 



Initiation 

Three primary factors which occurred somewhat concurrently contributed 
to the Initiation of the district *8 staff development, supervision and 
evaluation effort, a Program for More Effective Teaching (P>fET) • 

• The superintendent developed an Interest In 1977 and 1978 In apply- 
ing the Instructional Improvement and supervision staff development 
programs of Madeline Hunter and Ernie Stachowskl or. hearing their 
presentations at national conferences. He perceived a need to 
Improve the ef f'ectlv«nass and consistency of the Implementation of 
the existing teacher supervision/evaluation (TS/E) program. 

• A district teachers' association survey conducted In the latter 
half of the 1977-78 school year revealed that staff perceived sig- 
nificant Inconsistencies In teacher performance expectations and 
related teacher evaluation procedures In the district. Differences 
existed among principals and teachers, and among .-administrators at 
various levels within the district. 

• An attempt to rectify these inco^'*' '^tencles via shared, self-staff 
development sessions conducted In the 1977-*78 school year by dis- 
trict administrators for district administrators, using the Hunter 
films, ended In frustration due to insufficient content knowledge 
and training expertise (I.e., the administrative staff tried to use 
the Hunter £llms to train themselves In the principles of effective 
instruction with the Intent of applying this kna*?ledge to their 
class observations) . 

Planning and Start-up 

At the end of the 1977-78 school year, the super Intenaent and board 
decided that a more concerted effort was needed to address the district's 
TS/E problems, 

• They sent a committee of board members, subject area coordinators 
and administrators (N«'8) to study the exemplary Hunter-based staff 
development program of the Newport News, Virginia School District 

In October, 1978. This program had been underway for several years. 

• The comnittee endorsed the program and the board approved a pilot 
test of a similar staff development /supervision program for 1978-79* 

• An elementary school principal (former classroom teacher and teachers' 
association president) volunteered to be the district trainer and 
received extensive training in the "elements of Instruction" and 
"supervision" (Newport News, Virginia » 3-4 weeks In January 1979; 
and Stachowskl 1 week In 1979 at a NASE conference In Phoenix, 
Arizona). Bill Ethrldge, staff development director, Newport News, 
Virginia provided continuing support to the district trainer. 
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• The trainer conducted a highly successful six-day pilot workshop of 
Hunter's principles of effective Instruction for district central 
office staff, the elamentary and middle school principals, the teacher 
coordinators /supervisors, and two teachers selected by the teachers' 
association. In the spring of 1979. 

• The superintendent presented a three-year plan to the board for a 
Program for More Effective Teaching (PMET) In June 1979. Testimony 
from the pilot workshop participants and Bill Ethrldge was provided 
in support of the program. The board approved a three-year plan, 
and a few years later approved funds for the fourth year. 

• The teachers' association endorsed the staff development program 
contingent on the qualification that it not be tied to teacher 
evaluation. The teachers' association, apparently as the result of 
a difficult teacher strike a few years earlier, was mistrustful' of 
the administration's motives in Introducing the program. 

• The goals of the programs were: 

~ to improve teachers' Instructional skills, and develop a common 
vocabulary and set of expectations regarding the teaching act 
among teachers and administrators 

— to improve administrators' and supervisors' Instructional and 
supervision skills. 



Program Funding 

Annual Costs and Allocation of Monies 

The PMET program was first included as a line item in the district 
budget in 1979. The approximate costs per year since the program's 
inception were: 

1979-80 $30,000 

80- 81 $20,000 PMET Staff Development 

81- 82 $20,000 

82- 83 $20,000 



83-8A $10,000 Primarily non-PMET Staff 
8A-85 $10,000 Development 

The PMET costs cited above (1979-1983) were basically distributed as 
follows; 

90% = Salaries for substitute teachers to replace: (a) the 

district teachers undergoing five day« of training, and 

(b) two assistant trainers (teachers used in 1979-81). 
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5% 



Consultant costs and additional training for the trainer 
and district administrators. 



5% 



Materials and miscellaneous costs. 



Source of Monies 

In 1979 1 most teachers had reached the maximum allowable district 
reimbursement for graduate credits taken. Thus, the board-approved monies 
formerly targeted for graduate credit reimbursement were transferred to 
support staff development. In addition, in 1979 the district "director of 
personnel" position was changed, on the retirement of the incumbent 
director, to the "director of personnel and staff development." The new 
district trainer assumed the position ♦ Thus the change in Job description 
afforded the district a half-time staff developer or trainer without any 
significant expenditure of additional or new monies. 



Administrators; Content and Process 

A five-day summer training session was conducted in 1979 for district 
administrators and supervisors by Bill Ethridge of Newport News, Virginia, 
to provide administrative staff with clinical supervision skills. Ethridge 
was used because he had more credibility, at the time, than the district 
trainer who was the youngest administrator in the district. The training 
reviewed Hunter's elements of instruction and focused on a model clinical 
supervision procedure. During training, each administrator taught a practice 
lesson, scripted from 5-7 lessons, conducted 3-4 practice post-conference 
sessions and critiqued 3-4 post conferences of other administrators* In 
essence, administrators were taught how to gather and analyze classroom 
observation data, assess teaching act strengths and needs, set objectives 
for instructional conferences with the teacher, and plan and conduct the 
instructional conferences to provide reinforcement, coaching and follow-up. 
The supervision model adopted from the Newport News School District included 
the following steps: (1) inform teachers of expectations, (2) observe and 
record the teaching act, (3) analyze the record for strengths and needs in 
the teaching act> (4) prioritize the strengths and needs for the instruc- 
tional conference, (5) plan the instructional conference, (6) conduct the 
conference, and (7) provide follow-up. 

Most district administrators also received further training in the 
summer of 1980. They attended a 5-day training Institute sponsored by 
Madeline Hunter at either Villanova University (Pennsylvania) or B. Wallace 
College (Ohio) . 

The second, third and fourth years of PMET training also involved 
administrators, about a dozen times, in the two teacher-practice/obser- 
vation/post-conference sessions that concluded the PMET training of the 
teachers in their respective buildings. In the next-to-last session, they 
observed the trainer script and post-conference the teacher's lesson* In 
the final session, they scripted and post-conf erenced the lesson and 
received feedback on their performance from the district trainer* 



Staff Development; Content, Process and Evaluation 
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A series of in-^^er-;ice seminars were also conducted periodically by 
the district trainer for principals and supervisors during the first two 
years of the program to allow for more indepth discussion and understanding 
of the PMET instruction and supervision models 

Teachers: Content and Process 

Starting In 1979-80, teachers were trained in the PMET content adopted 
from similar programs in California and Virginia. Skills developed through 
PMET included: 

• how to select an objective at the correct level of difficulty 

• how to teach to an objective 

• how to monitor the progress of students and make adjustments if 
necessary 

• how to use, without abusing, the principles of learning: set and 
closure in the lesson, covert and overt behavior in learning, 
motivation of learning, and reinforcement, retention and transfer 
of learning. 

PMET was not designed to train all teachers to teach alike. Instead, 
it: (1) provided them with fundamental instructional skills necessary for 
the application of a variety of teaching styles, (2) enhanced their profes- 
sional decision making skills related to teaching, and (3) established a 
common instructional language enabling them to label and apply systemati- 
cally effective teaching techniques in order to improve instruction. 

The teachers were taught in small groups (N - approximately 15) ♦ The 
training for each group spanned a five or six-week period and involved the 
following. 

• Teachers were released for five full-day sessions at which the 
concept? of PMET were taught* Every possible effort was made 
through the use of illustrations, examples, questions, and 
activities to help teachers see the practicality of all that was 
advocated. 

• The instructor modeled the instructional skills presented in each 
input session. The teachers also viewed and evaluated videotaped 
segments of the components of effective teaching. 

• Time was provided between each input session (three or four school 
days) for teachers to practice the skills presented in their own 
classrooms. While in their classrooms, they were visited by a PMET 
instructor who observed a lesson and conducted an instructional 
conference following each of the five input sessions. 
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• Administrators accompanied the Instructor to practice sessions 
four and five and were Involved In the Instructional conferences 
following these lessons* 15o records of these lessons were kept by 
either the Instructor or the principal* 

• Lessons taught during the practice phase of the program had no 
bearing on a teacher's final evaluation* 

• After a teacher completed the course, the principal took over the 
role of observing and conferencing with the teacher to continue 
monitoring and reinforcing the Instructional skills taught. 

Evaluation of Staff Development 

The district conducted a formative evaluation of the PMET staff 
development program, both In the initial pilot session and in succeeding 
training cycles for administrators and teachers. Participants were asked, 
via questionnaire, for thair perceptions of the utility of the skills being 
presented, and for suggested changes in the training design and procedures* 
Participants' reactions to the training were generally quite favorable, 
although there was a fair amount of resistance at the secondary level* 
Most staff reported they had learned much that could be applied immediately 
to their teaching* Th3 district chose not to conduct any summative 
evaluation of the program per se* Staff were not tested formally for their 
acquisition of PMET content* In addition, no attempt has been made to tie 
teachers' subsequent classroom performance to student achievement data. 
Central office staff noted that such summative evaluation procedures would 
have been counter-productive to the "helping or school Improvement" 
philosophy of the program. 
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Staff Development Schedule/Tliaellne 



School 
Year 

(1) 1979-80 



Staff Trained 
In PMET 

Volunteer elementary 
and middle school 
teachers* 



Approximate 
Number 
Trained 

62 



Training 
Format 

Four five-week PMET 
training cycles 
(N=approxlTnatcly 15 per 
cycle) each involving 
five full-day input 
sessions • EacA session 
was followed by three 
days of practice in the 
classroom and a follow- 
up observation/coaching/ 
conference session with 
the teacher by the trai- 
ner or his assistant** 



(2) 1980-81 Remainder of elementary 
and middle school 
teachers 



Elve five-week PMET 
training cycles similar 
to the above 



(3) 1981-82 



(4) 1982-83 



Volunteer and non- 
volunteer high school 
teachers 

Rt^malnder of high 
school teachers » all 
t&achers new to the 
district, plus special 
subject teachers 
(librarians and 
guidance counselors 
received two days of 
training) 



65 Eour five-week PMET 

training cycles similar 
to the above 

70 Four five-week PMET 

training cycles similar 
to the above 



*Over the four-year staff development period, approximately 80% of 
the teaching staff volunteered for training* The remainder were recruited. 
There wei'e only four hold-outs ♦ 

**Two middle school teachers received two weeks of training at Newport 
News, Virginia and assisted the trainer as instructors and follow-up 
coaches in three training cycles in year two of the program. 
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iTppleinentatlon of District Supervision/ Evaluation Syst em 



Philosophy /Pol icy 

It Is the district's philosophy that both teacher supervision and 
teacher evaluation should be perceived by all professionals as a positive 
approach toward the improvement of Instruction and professional growth. 
The district employs a modified DEBE--333 procedure for teacher evaluation 
primarily for accountability and related personnel decisions. The PMET 
staff development program and the continuing follow-up supervision process 
are viewed as focusing primarily on teacher or school Improvement. The 
PMET supervision and DEBE-333 evaluation processes are seen by central 
office staff as "being related" or "informally connected." That is, 
specific PMET terminology does not appear on the DEBE-333. However, 
PMET-based supervisory classroom observation data, along with other 
Informal data, are used to support conclusions and ratings made in the 
course of the DEBE~333 annual evaluation process* 

Requirements 

The distrif^t requires two formal supervisory observations of tenured 
and four formal supervisory observations of non-tenured teachers each ye^r. 
The observations may or may not be announced, usually cover a full class 
period, and involve anecdotal record or script taping, lesson diagnosis, a 
post-conference 9 and a formal write-up. The observations are intended to 
focus primarily on PMET instructional concepts but may also focus on other 
aspects of teaching as needed (planning, class management, interpersonal 
skills, etc.). Teachers sign off on the formal observation reports and may 
comment in writing or request an appeal, if they so desire. 

The DEBE-333 evaluations are completed toward the end of the school 
year based on principals' informal and formal interactions with teachers 
and supporting data provided by the coordinators/supervisors. Copies of 
all formal supervisory observations and DEBE-333 evaluations are shared 
with the central office and kept on file in each building. The district 
has a due process procedure wherein, upon the receipt of an unsatisfactory 
rating for the year, the teacher has the right to a formal appeal procedure 
first to the principal, then to the superintendent and finally, if still 
dissatisfied, to the board of school directors. The district's agreement 
with the education association permits the district to freeze a teacher's 
salary increment for the next year if an overall unsatisfactory rating is 
received. Only two unsatisf actories have been given in the last five years 
and the district has not had occasion to counsel-out any teachers. Two 
unsatisf actories in a row constitute grounds for dismissal. 

Organization and Process 

The district's supervision and evaluation process is carried out by 
the school principals, assistant principals, and the eight subject area 
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coordinators/supervisors in che district. The superintendent and assistant 
superintendent manage the process and only become directly involved in 
observing teachers in cases where permanent Substitutes are being 
considered for tenure-track employee status or in situations where there 
are serious questions about a teacher's competence'. Principals and 
supervisors are expected to spend about 20% of their time on 
supervision/evaluation, while the superintendent and his assistant spend 
from 10% to 20%, 

The three elementary school principals^ teacher observation load 
ranges from approximately 15 to 25 teachers and specialists per year* The 
middle school principal and assistant have the largest load, splitting the 
observation of some 60 teachers and specialists per year» approximately 
half of whom are also observed by a coordinator /supervisor . The high 
school principal and his two assistants divide the observation of some 70 
teachers, almost all of whom are also observed by one of the 
coordinators/supervisors^ 

System Monitoring 

The superintendent and his assistant monitor the implementation of the 
supervisory observation/conferencing system by sitting in on one teacher 
observation/conference with each of the district administrators and 
coordinators/supervisors each year* They provide feedback and 
reinforcement as needed • They also review copies of the written 
observation reports to make sure they are focused on the elements of 
instruction and in compliance with the prescribed district reporting 
fo:.TDat. The system provides some latitude for individual differences in 
the various steps in the supervisory model. Generally speaking, central 
office staff are satisfied that the PMET training has brought a 
satisfactory measure of standardization across schools and classrooms to 
the district teacher supervision process. The DEBE-333 procedure is done 
to satisfy state requirements, and would probably be dropped if district 
officials had their choice* 



Administrators^ Perceptions 

The high school principal (11-12), assistant high school principal 
(9-10), middle school principal, three elementary school principals and two 
subject area coordinators/supervisors (7-12) were interviewed for their 
perceptions of the district's teacher supervision/ evaluation system. 

Relationship Between Teacher Supervision and Teacher Evaluation 

The administrators indicated that the district "attempts" to make a 
clear distinction between their role as Supervisors and evaluators. They 
noted* however, that the two roles and procedures of necessity tend to run 
or blend together • That is, they are required to write-up the formal 
teacher observations (supervision) and do use these reports as data to 
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assist them in completing the DEBE-333 end-of-year evaluation. Most 
indicaued they had no problem in implementing their dual role. Two staff, 
however, felt they might be more effective at improving instruction if the 
roles could be more cleanly separated and if they could find more time to 
conduct informal observations and conferences (i.e., no write-ups). 

Utility of PMET and Clinical Supervision Training 

Without qualification, all staff found the PMET and clinical 
supervision training to be very effective and valuable. All but one, 
however, expressed a need for additional training. Half expressed interest 
in additional training in one or more areas related to observing and 
conferencing teachers. Two expressed a desire for a support group to meet 
regularly to discuss system implementation /refinement skills and problems. 
Two others desired training in other content areas related to teaching 
(e.g., class management, higher order thinking skills, etc.). 

Feasibility of Implementation 

The amount of total time per year staff estimated they spent on formal 
teacher supervision and evaluation varied from approximately 35 hours per 
year to 140 hou^rs per year. The estimated average time to complete one 
formal observation-report preparacion-post-conf erence cycle also varied 
between staff from about one to two hours. Half of the staff interviewed 
reported they had problems finding and scheduling the time for the 
observations. All staff, however, felt more comfortable in implementing 
their roles as supervisors and evaluators after having experienced the 
staff development program. All were also able to cite specific ways in 
which they assisted specific teachers to improve their instructional skills 
(e.g., suggest different teaching techniques, interpreting class reaction, 
coaching on PMET concepts). 

Impact of Supervision 

Administrators generally perceived that PMET helped teachers improve 
tiieir instruction and increased their awareness of whether true learning 
was occurring* They indicated that coramcn expectations, a common language 
and a supervisory structure were now in place. Two felt, however, that 
even more inservice, follow-up coaching or training could be conducted to 
deal with the small percentage of teachers who are not as effective as they 
might be* They noted there has been little PMET inserv icing and follow-up 
in the past two years. 

Administrative staff also shared the common perception that PMET was 
more favorably received in the elementary school. There was both 
administrative and staff resistance to PMET at the secondary level. The 
initial reactions of the secondary staff to PMET training were negative. 
Although secondary staff are less negative now, and a good number have come 
around, problems still exist. Several felt some of the problems could have 
been ameliorated if the program had been Introduced differently, perhaps in 
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a less unilateral fashion with illustrative R&D dat^, statistics and 
testimony provided to support its use. Secondary staff's mistrust of the 
former superintendent's motives in introducing the program (i,e,» "a way to 
get us»" "more work") » also purportedly confounded its introduction* 

Impact of Evaluation 

Most administrative staff questioned the extent to which the DEBE-333 
end-of-year evaluation helped teachers to improve their instruction. They 
perceived the during-the-year observations to he more useful to that end, 
They did feel it: (1) served as a wrap-up and overall summary of a 
teacher's performance, (2) served as a source of recognition^ (3) provided 
one-to-one contact, and (4) served as a vehicle to track a teacher's per- 
formance from year-to-year. They perceived that teachers probably had no 
strong views toward it one way or the other and viewed it as "so-so" or a 
"pro forma" procedure, except in those few cases where negative ratings 
were received* Most also perceived that outstanding and poor teachers 
would not get similar ratings under the present system, although average 
and good teachers might get similar ratings. Overall, they noted it had 
less impact than in the past because of its decreased significance for 
continued employment. 

Perceived Strengths 

Adminisitrative staff agreed that the most important contribution of 
PMET was that it established a common language and common expectations 
among staff as to what comprises "good teaching," It also helped increase 
communications* Another perceived strength was that administrators felt 
more comfortable in evaluating lessons in subject areas in which they were 
not content experts* 

Perceived Problems 

Most administrators reported that finding the time to schedule the 
required observations /conferences was a problem, A few felt the program 
was beginning to lose some of its impact, in part due to tho increased 
workload of administrators, which reportedly hindered their efforts to 
carry out the required observations /conferences "as well as they would 
like," A few also questioned the present level of commitment of the school 
board to support and follow through on the program as a district priority. 
Several also perceived some evidence of continued resentment and misunder- 
standing of the purpose of the PMET supervision process on the part of 
teachers and attributed this issue to the way the program was introduced. 

Perceived Needs 

Each of the following were perceived as needs by two or three 
administrators: 

• to assess the district's teacher observation reports annually to 
determine common instructional needs and design follow-up inservice 
sessions for teachers to address these needs (e*g,, "There's a need 
for follow-up inservice of teachers on PMET*") 
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• to continue to provide Inservice to administrators to upgrade their 
observation and conferencing skills ("My quality needs to continue 
to grow." **I have trouble recommend Ing ways to help average 
teachers grow." "The once a year taonltorlng of my conferencing 
skills by central office staff hasn't influenced me much/') 

• to have the administrators meet periodically to share ideas and 
discuss common problems regarding supervision and evaluation ("We 
did this a little in the first few years, now we don^t do it 
anymore.") 

• to receive more administrative help with supervision and 
evaluation 

• to concentrate more assistance on those teachers who really 
need help 

• to explore the use of additional data gathering procedures 
(e*g*, time-on-task, teacher expectations, etc.) 

• to bring the instructional sections cf the DEBE-333 more in line 
with the teruinology used in PMET and to make the other sections 
of the DEBE-333 more objective or low- inference in nature. 



Teachei3^ Perceptions 

Forty-two teachers were interviewed to elicit their perceptions of the 
district's supervision/ evaluation system* Nineteen taught at the 
eilementary level, 12 at the middle school, and 11 at the high school « 

Teacher Supervision and Teacher Evaluation 

The majority of the teachers (over 80%) did not perceive a clean dis- 
tinction between the purpose of teacher supervision and teacher evaluation. 
They saw the procedures (process and format) as different but felt they 
blended together and both resulted in or constituted evaluation. The fre- 
quency which teachers reported being observed and their descriptions of the 
procedure (i.e«, announced or unannounced observations, review of planbook, 
anecdotal notes of lesson observation, post-conference critique of lesson 
baaed on analysis of notes — reinforcement of positives and suggestions for 
changes — followed by a written observation report which they sign off on) 
were in full agreement with the information provided by the administrators. 
In a half-dozen instances, the teachers reported that the final observation 
report was prepared prior to the post-conference. All teachers also 
appeared to have a clear and common understanding of the expectations for 
the class observations (i.e., to check on the implementation of PMET instruc- 
tional concepts as well as planning, class atmosphere, student reactions, 
discipline, etc.). In general, the elementary teachers reported a much 
higher level of informal supervision (informal class drop-ins and regular 
conversations/interaction with the principals) than the middle or high 
school teachers. 
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All teachers were familiar with the district's modified DEBE-333 end- 
of-year evaluation form and reported either receiving copies of it and/or 
having a conference with the principal regarding their overall performance* 

Satisfaction With Input 

Most of the teachers perceived th'-c they had little or no say in the 
design of the PMET system. This did not particularly disturb teachers at 
the elementary level, but did disturb several secondary teachers who 
reported they had mistrusted the former superintendent's intention in 
instituting PMET« Several teachers (nine or ten) acknowledged the 
teachers' association's involvement in adopting PMET but again felt that as 
individuals they did not have much say in its adoption (e^g., "It was the 
superintendent's decision*"). However, the teachers were of the opinion 
that most staff have now accepted PMET and that much, but not all, of the 
resistance at the secondary level has dissipated* 

Utility/ Impact of PMET /Super vis ion 

When asked about the utility of the PMET training and practice 
sessions they had experienced, almost all teachers commented favorably on 
its usefulness* Sixteen teachers gave specific examples of how it had 
helped their teaching and/ or of how they do things differently now ("I now 
give a lesson rationale, objectives, and include review*" "I pay more 
attention to lesson structure." "I focus on one skill at a time," "I use 
different questioning techniques*" "I now check more for understanding and 
provide closure*" "I picked up many useful tools*"). Over half of the 
teachers also reported that it made them "more aware" of the teaching 
process and that they felt "more organized and structured" in their 
teaching* Ten teachers also indicated that the feedback they have received 
after class observations by principals and/or coordinators/supervisors has 
been helpful. A few teachers thought that it was just a review of what 
they had already been doing and thought it would be more useful to new 
teachers. Only two indicated that it was not helpful* 

Teachers were also asked, "What percent of the total knowledge and 
skills that you currently employ in teaching would you attribute to: 
(1) your practice teaching experience, (2) trial-and-error learning on 
your own over the years and/ or interaction with peers, and (3) your PMET 
experience?" The majority of the teachers attributed the acquisition of 
more than half of their total knowledge and skills to trial-and-error 
experiences on their own and to interaction with peers. The approximate 
percent of total knowledge and skills attributed to PMET was as follows: 
about 50% of the teachers attributed 10% of their teaching knowledge and 
skills to PMET, another 30% attributed 20% to 25% to PMET, a quarter of the 
teachers attributed 35% to 40% to PMET, and one person attributed 50X of 
their teaching knowledge and skills to PMET. In short, close to 60% of the 
teachers attributed 20% or more of their teaching expertise to their five 
or six week PMET staff development experience* 
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Utility/Impact of Evaluation 



The great majority of the teachers did not view the DEBE-333 etid-of-year 
evaluation as helping them to improve their teaching • They saw it as a pro 
forma exercise. A few teachers commented that it: (1) "might help new 
teachers," (2) "provides positive reinforcement at the end of the year," 
(3) "might keep people on their toes," and (4) "serves as my grade card." 
A few teachers also stated that they 'Mid not even bother to look at them" 
or that they "had a lot of laughs over them in the teachers' room." 

System Fairness 

When asked whether they perceived the teacher supervision/evaluation 
system as fair, approximately a third said it was, a third said they were 
not sure and thought there might be some inconsistencies, and a third 
thought that it should be tightened up so there was more accountability. 
The latter group did not think it was fair that all teachers received the 
same rating. They felt the system should be modified to recognize and 
acknowledge the difference between above average and average teachers, and 
deal more directly with unsatisfactory teachers. 

Strengths 

At least half of the teachers perceived that the major strengths of 
the PMET training/supervision process were that it helped them to 
systematize and organize instruction, and provided a common language to 
facilitate discussion of teaching. At least a quarter cited one or more of 
the following as strengths: the program increased their awareness of 
effective teaching, it provided for better comiminication between teachers 
and principals, and/or it clarified the district's expectations regarding 
supervision and evaluation. 

Needs/Problems 

The most commonly identified need or problem (mentioned by about eight 
or nine teachers) was that the program, which they felt was forced on them, 
was misrepresented. That is, they felt it was a program that was proposed 
to help them and instead, was later used to evaluate them. Other concerns, 
each ci^'ed by three or four teachers, included the following* "We could 
use a refresher on PMET." "Two observations per year are not sufficient to 
evaluate us." "The observations should contain more positive reinforcement/* 
"The principals in the middle school and high schools need to be more 
visible." "The supervision process should provide more help and follow-up 
than it currently does." "Some teachers put on a show and only use PMET 
when observed." 

Additional perceived problems were each cited by one or two interviewees. 

• "it's unfair that all the evaluations end up the same. There are 
some incompetent tenured teachers that the administration should 
confront and shape up.'* 
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• "The principals should teach a few classes to increase their 
credibility and keep in touch with the changing times and the 
changes in students/' 

• "The DKBE-333 and PMET observations should be more closely related 
and use similar terminology." 

• "Principals should not announce their visits, instead they should 
drop in without notice." 

District Administrative Staff Overall Reflections on 
Program Design, Operation, and Impact 

Perceived Strengths 

When asked about the strengths of the district teacher supervision/ 
evaluation system, administrative staff indicated the following. 

• A common language about instruction has been developed and 
instructional skills have been made the focus of attention of both 
teachers and administrators. 

• The level of trust and communication between teachers and 
administrators has increased to some degree. 

^ The PMET procedure has helped teachers by providing specific 

feedback and positive reinforcement related to the things they are 
doing well and has identified areas in which teachers need 
improvement . 

• Administrative staff feel that they have a tetter handle on in- 
struction, observation and conferencing. 

• The modified DEBE~333 evaluation form, which .was jointly developed 
and agreed on by the administration and teachers* association, is 
better than what was used before. 

Perceived Problems 

The major problems faced during the initial start-up snd 
implementation phases of PMET were: overcoming the resistai.ee of 
administrators who questioned both the intrinsic value of the program and 
where they could find the time for it, overcoming the resistance of 
teachers who saw the program as evaluation oriented and out-to-get~them, 
scheduling teachers in pairs related to their content areas for PMET 
training, and finding qualified substitutes to take over their classes 
during training. Administrative staff stated that part of the initial 
resistance was overcome Just by doing it, by getting the program going. It 
also became part of the administrator's MBO's and that helped to decrease 
their resistance. It was noted that there were no particular problems 
initially with the board in obtaining funding due to the former 
superintendent's work with the board in selling the need for training. 
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Administrative staff perceived, however, that a problem still exists 
in breaking dovn the feeling that administrators and teachers are on 
different sides* They noted that administrators are still perceived by 
some teachers as engaging in "snoopervislon" as opposed to '^construct ive 
supervision." They also felt that the district or the TS/E system was not 
doing as much as it should with regard to following--up on teacher 
observations in those cases where the observations revealed that the 
teachers could use some assistance* The dilemma they faced was how to do 
this without neglecting the observations of the bulk of the teachers whose 
Instructional skills and general performance was perceived to be up to the 
district's expectations* Altho»'gh they were aware of this problem, they 
had not devised a solution. They also perceived that the TS/E system, as a 
whole, was in need of renewed attention as a district priority in that the 
quality of implementation (observations, etc.) had lost some ground in the 
past few years when the district turned its attention to priorities other 
than PMET after the last PMET training cycle. 

When asked about problems with the DEBE-333 evaluation procedure, 
administrative staff acknowledged that: (1) parts of it were still 
subjective or judgmental, and that the ratings and interpretations of the 
evaluation categories were subject to varied interpretations depending on 
who was doing the rating; (2) it does not permit them to differentiate 
between what is acceptable teaching and what is outstanding teaching, both 
types of teachers get the same "satisfactory" rating; (3) differentiated 
ratings of teachers on the DEBE-333 are not legally meaningful any more and 
it has lost its impact; (4) It does not play an important role in helping 
teachers to improve, it is a pro forma exercise; and (5) they would like to 
develop a more meaningful teacher evaluation procedure, one based in part 
on PMET and in part on recent effective schools and classrooms research. 

Changes Made or Things They Might Do Differently 

In the course of implementing PMET, district staff made the following 
changes or adjustments. In year one, the training for teachers was cut 
from six to five, input days. Additional training for principals was 
provided, in year tvo of the program, by involving them in observing and 
conferencing the fourth and fifth teacher practice sessions. The training 
materials were supplemented by developing a library of five or six video 
tapes using district staff to model effective instructional procedures. 

When asked what they would do differently if they had the opportunity 
to do it all over again, administrative staff indicated they would: 
(1) begin by implementing the program K-12 from the start and mixing staff 
from all levels during training; (2) provide for more involvement of the 
secondary teachers in the original conceptualization of PMET, to give them 
a sense of ownership; (3) build in more teacher observations and 
conferences that did not involve a write-up, because the observations are 
perceived as evaluation when they are written up; and (4) Incorporate a 
much more meaningful end-of-the-year evaluation based on more relevant 
criteria and tied more directly or overtly to the observation reports and 
PMET. 
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Assistance They Could Have Used 



Staff reported that they could have used more well-developed training 
materials both for teachers and administrators. They noted that appropriate 
training materials were not readily available then and are not available 
now. They mentioned using some of the Hunter films. The problem they had, 
however, was that the examples of instructional techniques included in the 
films were not appropriate for all grade levels. Based on their initial 
experiences with the films, they decided to do demonstration teaching of 
the concepts on their own and not to use the films. 

Advice to Others 

When asked what advice they had for other districts considering the 
development of similar teacher supervision programs, they offered the 
following suggestions and cautions. 

• One of the major keys to the success of the program is the 
commitment of the board, central office staff and the 
administrators. 

• Involve teachers from the beginning in the planning and conceptual-- 
ization of the program. 

• Do not begin training until you have in-houae experience. The 
people who are to be the district's trainers need extensive train- 
ing. 

• Plan for at least a three to five-year commitment ♦ 

• Explore several different programs before you adopt one. It is 
important that the district commit to one instructional model, 
though, and in our case we chose Hunter. 

• Do not expect a panacea. No one program has all the answers. 

• Do not expect toe much too soon. It takes time and money to 
develop a trainer's skills and subsequently to develop the staff's 
skills. Do not rush into it too quickly. Give yourself a reason- 
able timeline. 



Summary of Findings and Discussion of Implications/Issues 

Summary of Findings 

The efforts of the Abington Heights School District to: (1) improve 
administrators' and supervisors' instructional and supervisory skills, (2) 
improve teachers' instructional skills, (3) develop a common language of 
instruction, (4) establish clear criteria and common expectations regarding 
instructional supervision and teacher evaluation, and (5) upgrade the 
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consistency of teacher supervision and evaluation, have in large part been 
successful. Varied levels of progress appear to have been achieved on all 
of the above fronts* In addition, staff generally reported increased 
levels of communication between administrators and teachers. 

Some of the factors or strengths of the system and the procedures used 
to set the system in place, that appear to have contributed to that success 
Include the following ♦ 

• The district informally explored alternative instructional models, 
involved the board in the process and obtained its support from the 
beginning, selected a single R&D-based model via a representative 
group process which involved most all vested groups in the decision, 
spent time site-visiting an exemplary implementation of the model, 
utilized experienced and credible consultants, conducted a pilot test 
of the model to test its efficacy and acquire key advocates for the 
model, established a three to four-year timeline for development 

and implementation of the system, allocated adequate resources to 
support the project, selected a staff member who had the trust and 
respect of a majority of the district's teachers to be the district's 
trainer, and provided the trainer with adequate training. 

• Regarding staff development and system implementation, the district: 
provided training in the elements of the model for both adminis- 
trators and teachers, provided for coaching of administrators and 
teachers during training, established support/ seminar groups for 
administrators in the initial years of system implementation, kept 
no formal observation records on teachers' performance during 
training; » conducted formative evaluations of training sessions, 
required that principals' post-training teacher observations involve 
anecdotal note taking/lesson analysis and post-conferences followed 
by a written report in a structured format, annually monitored 
principal's observation-conferencing behaviors and provided feed- 
back on their performance, required a specific number of observa- 
tions of each teacher each year, and periodically reviewed 
principals'/supervisors^ observation reports for instructional 
focus and quality. 

No system is without its problems. Some of the problems perceived by 
district administrative staff, teachers, and/or the researchers, which 
appear to have constrained or placed a ceiling on the considerable level of 
success achieved by the district in updating its teacher supervision/ 
evaluation system to the end of improving instruction are best summarized 
as follows. 

• At the time the program was conceptualized, the district, con- 
strained by the after effects of a recent strike and concomitant 
pressures from the teachers* association, did not clearly concep- 
tually relate teacher supervision for instructional improvement to 
the annual teacher evaluation or rating procedure. In fact, the 
district administration agreed to keep the two procedures separate 
with regard to both their operational procedures and intent. Given 
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the dual supervisory evaluative role of principals, the design and 
schedule of PMET training and evaluation implementation ♦ and the 
inherent overlap of the teacher performance domains involved in 
teacher supervision and teacher evaluation, this appears to have 
been an almost impossible task. Consequently, although admini- 
strators and teachers share a common understanding of the proce- 
dures involved in supervision and evaluation they differ to a large 
extent in their understanding of the purpose and intent of the two 
procedures; with administrators professing to see a clearer dis- 
tinction between the purposes of supervision and evaluation than 
teachers. A number of teachers felt that a trust was abused in 
that they perceived that the purpose of PMET was misrepresented, 
and that contrary to the district's initial statement, PMET was 
being used to evaluate teachers. 

• Other factors which were perceived to have constrained the 
effectiveness of the program were: the secondary staff could have 
been more actively involved in the start-up of the program; elementary 
and secondary level staff were trained separately, thus maintaining 
traditional political units and differences between these units; the 
amount of time allowed for principals and teachers to practice PMET 
skills was perhaps not long enough to build sufficient trust; more, 
informal observing/ conferencing, with no formal write-ups would 

also have contributed to greater levels of both skill and trust; 
the long-term needs of principals for additional training, support 
and follow-up monitoring/ coaching were addressed by the district in 
a modest fashion; the district PMET trainer did not have a strong 
role identification with the secondary staff; follow-up work with 
teachers who really need help has not been perceived as intense or 
effective; a level of perfunctory implementation of the obser- 
vation/conferencing process has crept into the system over the past 
few years due to time constraints and other factors; the modified 
DEBE-333 evaluation procedure is perceived as a pro forma exercise; 
and, at least a third of the staff do not feel that the teacher 
supervision-observation system has any teeth or accountability 
built into it and would like to see It modified to afford 
recognition of outstanding versus average teachers, and to deal 
firmly with ineffective teachers. 

Implications/ Issues 

The experiences of the Abington Heights School District offer many 
hints, in terms of both do's and don'ts, to other districts considering the 
adoption of similar teacher supervision/ evaluation systems. These are 
implicit in the findings reported above. Some of the most salient 
imp 1 ica t ions inc lud e : 

• the importance of using a representative group process in program 
selection and the importance of providing quality staff training. 
The use of a representative process to select a supervisory 
instructional model (broad-based committee involvement, site visits 
to exemplary programs, use of consultants, pilot test to build an 
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advocacy group), and the provision of a quality training program 
for both administrators ^nd teachers in a single model of 
instruction (input, modeling, guided practice, feedback and 
follow-up sessions) , are factors which contribute tc program 
success 

the necessity of clarifying fully, up-front, the relationship 
between instructional supervision/staf f development and teacher 
evaluation. Ideally, the two should be related and after a 
sufficiently long training and phase-in period (one to three years 
may be required) the instructional model chosen by the district and 
the related observation/conferencing procedures should become a 
part of teachers' and administrators' accountability, respectively, 
for annual evaluation purposes 

the importance of taking sufficient tinie and expending sufficient 
resources to sell the program to all school staff to win their 
advocacy and support of the program 

the desirability of mixing staff from all levels (K-12) during 
training to create new political units in support of the program^ 
reduce traditional program, role and turf issues/barriers, and 
facilitate cross^level communications 

the desirability of providing a sufficiently long program training 
and phase-in period for all staff before the program (new knowliidge 
and skills) is made a "formal" part of teachers' observation and 
evaluation* This not only increases staff knowledge and sUllTs, it 
also builds trust and confidence that the district intends to provide 
more than adequate practice, help, and support before staff are 
formally held accountable for mastering new teaching skills 

the importance of providing quality, sustained, long--term 
monitoring of system implementation along with similar follow--up 
and support for all role groups involved (principals, supervisors 
and teachers) to monitor and adjust the system to Insure Its 
continued effective and efficient operation. 
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CASE STUDY 2 



East Penn School District 
6A0 Macungle Avenue 
Euunaus^ FA 18049 
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CASE STUDY 2: EAST PENN SCHOOL DISTRICT 



Background Data 

Schools 



• 9 Elementary Schools 

5 (K^3) 
2 (K-5) 

• 2 Junior High (7^8) 

• 1 High School (9-12) 

Staff 

• Superintendent 

• Director of Curriculum and Instruction 

• 5 Elementary School Principals 

• 2 Junior High School Principals and 2 Assistant Principals 

• I High School Principal and 2 Assistant Principals 

• 5 Curriculum Supervisors (K-12) (English, Math, Science, Health and 
Physical Education, and Art). 

• 7 Department Heads or Subject Area Leaders (non-supervisory: serve in 
departments where there are no supervisors) 

• 1 Staff Development Person Oi time) 

• 130 Elementary School Teachers 

• 170 Secondary School Teachers 

(71% of staff » 10+ years of experience) 

Average Per-Pupil Expenditure (Total Budget - Total Students) 

• $3»500 

Teacher Supervision /Evaluation 

• Modified DEBE~333 procedure, instituted in 1981- 

• R&D-based supervisory staff development program for principals, and 
ProCEED staff development program for teachers (and principals) • 

• Teacher evaluation has been included in the collective bargaining 
agreement with the PSEA affiliated teachers* association for the past 
15 years. 

Community 

Located five miles from Allentown, Pennsylvania, the East Penn School 
District lies in the heart of the Lehigh Valley, a metropolitan area with a 
population of a half million people. Extensive business, industrial, and 
residential development is taking place amid expansive and flourishing 
agricultural lands* The 50 square mile district, which has a population of 
33,600 residents is composed of five municipalities. The population, which 
has doubled in the past 20 years, ranges from middle to upper middle class* 
Housing consists of mostly single homes in suburban communities. 




BEST COPY AVAILABLE 



Students 

• 5,800 students 

• 99.9% white 

• Secondary student 
population slightly 
declining, elementary 
population growing 
slightly 




Program Initiation^ Planning, and Start-up 

Initiation 

The Initiation of recent efforts to ■'.mprove the teacher supervision/ 
evaluation system In East Penn can be traced back to 1980-81 ♦ The superin- 
tendent at that time established a Teacher Evaluation Coamlttee to review the 
teacher evaluation process. There had been some discontent in the district 
over implementation and communication problems related to teacher evaluation 
that had developed after a 22-day teacher strike in the mid-1970s ♦ The 
Teacher Evaluation Committee, composed of teachers r administrators, students, 
citizens, and board members, worked throughout the 1980-81 school year and 
developed a modified DEBE-333 evaluation procedure* The cotomittee also put 
together a notebook. Tips for Tei^chers , which included a variety of R&D-based 
resource materials for teachers^ use in improving instruction. This 86-page 
loose-^leaf compendia addressed the teacher as a professional and included 
resources related to: organizational skills, working with both normal and 
exceptional children, the steps in the teaching act, questioning techniques, 
student motivation, discipline and evaluation, all aspects of school 
communications, videotaping guidelines, and the new evaluation procedure* 

Planning and Start-up 

Two sets of activities need to be described in this section* Following 
the 1980-81 Teacher Evaluation Committee work, the superintendent attempted to 
upgrade the quality and consistency of administrators' teacher observation/ 
evaluation skills • In--house administrative staff, experienced in the Dick 
Manatt model of teacher supervlsion/evaluatlon, conducted summer seminars/ 
training sessions on that model for the district administrators in 1981, 1982, 
and 1983. 

Paralleling the above activities, the district established a staff 
development committee to plan a comprehensive multi-year staff development 
program to expand and upgrade teachers' instructional skills* The initiative 
was part of the district's 1982-83 long range planning effort* Committee 
staff of teachers' and administrators worked from December 1982 through May 
1983* . They visited and examined several other districts' staff development 
programs and reviewed research findings on effective teaching* They also 
reviewed district needs assessment data, past Inservice committee reports, and 
teachers' evaluations of the previous year's district inservice activities. 
Based in part on appraisals of the above input, the committee recommended the 
content and format for a new program. Project ProCEED (Professionals Committed 
to Excellence in Education)* The proposed content Included the development of 
six instructional modules on various topics. The topics chosen did not 
Include the Hunter model; staff decided against it. 
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The board approved the program and appointed two staff development 
co"dlrecCors In June 1983, From June 1983 to January 1984, the co-directors 
and various district staff (teachers and administrators) developed the 
program* The development sequence Included: (1) formation of the ProCEED 
committee and recruitment of presenters; (2) committee development of six 
Instructional modules on brain development/ learning styles, teacher 
expectations and student achievement. Bloom's taxonomy, students with special 
needs, time/classroom management, and student motivation, respectively; (3) 
final recruitment of presenters and continued planning of the sessions; (4) 
recruitment of participants (teacher volunteers); aad (5) pilot testing and 
revision. 

The six Project ProCEED modules that were developed have been used since 
February 198A to train school staff. ProCEED Is characterized by: (1) In- 
house content development, (2) use of ln--house presenters (administrators and 
teachers); (3) modeling of effective Instructional techniques by presenters; 
(4) homework assignments for the participants to Induce application of the 
content/skills Imparted In each of the modules; and (5) released time for 
volunteer teachers to participate In the six full days of ProCEED training. 

Work on Project ProCEED occurred In a period of district leadership 
transition. The former superintendent resigned In January 1983, and an acting 
superintendent was In place for eight months. The current superintendent took 
office In October 1983. The new superintendent perceived that a number of 
very positive activities were occurring In the district. He also decided, by 
June 1984, that there were a number of loose ends that needed to be addressed 
and/or coordinated (i.e., the consistency or reliability of the teacher 
supervision/observation process; the role of the district supervisors whose 
assignments had recently been expanded to Include the elementary school; 
problems In communication among and between administration and teachers; lack 
of clarity /misconceptions In communicating the district's expectations regard- 
ing supervision of Instruction to teachers; the need for more feedback to 
teachers; a need to review and monitor the supervision process; and, a need to 
clarify the relationship between Project ProCEED and teacher supervision/ 
evaluation) . 

The superintendent hired a private consultant, Harvey Silver, to assist 
the district with some of the above problems In the summer of 1984. The 
start-up of the district's efforts to refine and coordinate existing teacher 
supervision/evaluation activities and district staff development activities 
commenced In the summer of 1984. Currently, the district Is still engaged In 
this effort. The superintendent's goal Is to bring all the pieces together In 
the 1985-86 school year. The pieces consist of additional ProCEED training 
for teachers In "an Instructional model"; additional training for admini- 
strators; clarification of the district's expectations of teachers regarding 
effective teaching and concomitant observation/evaluation procedures; and, an 
effort to develop a common language about, and more standard procedures for, 
the district's supervision program. 
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Program Yundlng 
Annual Costs and Allocation of Monies 



The staff development costs cited below relate to the district's efforts 
to upgrade teachers' instructional capabilities via Project ProCEED 
(1983-present) and administrative staffs' supervisory skills as of 1984, 



Project ProCEED 
Summer 1983 

School year 1983-84 

Summer 1984 

School year 1984-85 



$ 2,500 Salaries/supplies for program 
development 

$ 4,800 Salaries for substitutes for teachers 
taken out of class for training 

$ 9,000 Staff developer (Jg time) 

$ 1,500 Summer planning 

$14,000 Substitute salaries (see above) 

$ 1,200 Supplies 

$13,000 Staff developer (4 time) 



Administrator Training 

School year 1984-85 $ 5,400 Consultant fees 



Source of Monies 

Staff development efforts in East Penn have been funded by both district 
and state monies- In the 1984-85 school year, for example, $8,000 of the 
$10,000 state grant for instructional improvement was applied to fund Project 
ProCEED. The remaining monies for staff development since 1983 have come from 
regular district funds. The superintendent noted that with a total district 
budget of $20 million ($12 million = salaries), monies can certainly be found 
for staff development. The monies can come from cutbacks /savings on different 
projects (e.g., cutting back a little on a driveway paving program) • It can 
also come from monies budgeted/projected but not spent due to retirements, 
sabbaticals, maternity leaves, etc. Unspent monies allocated for staff 
tuition reimbursements might also be applied to staff development. In 
essence, a district can find money for staff development if it wants to. 
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Staff Development: Content, Process and Evaluation 



Administrators: Content and Process 

District administrators were exposed to four half days of training In the 
Dick Manatt model of teacher supervision/evaluation In the summer of 198 !♦ 
Training consisted of practice In Kanatt^s chronological lesson progression/ 
description technique, analysis of videotaped lesson segments, comparison/ 
discussion of the ratings of the videotaped lessons, practice In Interviewing/ 
conferencing techniques, and discussion of the use of the district's new 
teacher observation form* Administrators received further practice and /or 
review of the Manatt techniques In three half-day training sessions in the 
summers of 1982 and 1983. In the course of the above training, the adminis- 
trators were also exposed to several models of teaching (e.g., Berliner's 
prescriptions on lesson organization, DeCecco's components of a problem 
solving lesson. Hunter's steps In the teaching act, and research on effective 
teaching). However, the district did not adopt one model of teaching or train 
staff In-depth In any given model. 

In the summer of 1984, the district brought in a consultant, Harvey 
Silver, for five days to work with the administrative staff on a number of 
Issues related to TS/E. The consultant put the administrators through several 
exercises to assist them in understanding their administrative styles and 
learning styles, and the Implications of such for their teacher observing/ 
conferring. The consultant also modeled a variety of instructional techniques 
and strategies (concept formation, problem solving, advance organizers, and 
different types of thinking) related to increasing the effectiveness of lesson 
design and instruction. Part of the consultant's modeling was based on 
Madeline Hunter's model of effective teaching. 

Pollowing the summer session the consultant spent five days in the dis- 
trict, spread over the 1984-85 school year, meeting with small groups of 
staff. The Intent was to follow-up on staffs' application of the content 
presented in the summer; engage in individual problem solving in specific 
areas of uncertainty; and develop action plans to addresss those uncer- 
tainties. Overall, the superintendent felt that the district's interaction 
with the consultant met with moderate success. 

Teachers: Content and Process 

Since 1981, staff development for the teachers has consisted of their 
exposure to Tips for Teachers , produced by the district-wide Teacher 
Evaluation Committee in that year. The district has also covered various 
instruction related topics each year, in the annual two days allocated for 
teacher Inservlce. Tbe district's primary staff development thrust to Improve 
the quality of instruction, however, has been Project ProCEED. Project 
ProCEED training began in the second half of the 1983-84 school year. 
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Teachers have been trained in ProCEED in small groups, 20-23 K-12 staff 
per group. Each group has also included a few administrators. The first 
group received training in the second half of the 1983-84 school year. Three 
groups were trained in the 1984-85 school year. Three groups each year are 
also scheduled for 1985-86 and 1986-87, The program will continue with the 
expectation that most or all of the staff will eventually have participated, 
Grou? 1 was trained over a six-week period with one day of training per week. 
With groups 2-4, two weeks were scheduled between each day of training to 
afford participants more time to complete related homework assignments. 
Sessions may be scheduled every third week for groups 5-10 to allow even more 
time for assignment completion and content practice. 

To date, approximately 65-70 teachers and some 15 administrators have 
been trained in the 6 ProCEED modules. The training, which has the full 
cooperation and endorsement of the union i has been quite well received. 
Follow-up meetings and group reunions have also been scheduled to provide 
support for staff who have been through the program. Opportunities have been 
provided and are planned to bring people back together by groups, topics 
and/or departments to discuss/extend their ProCEED training. Overall, the 
district sees ProCEED as the beginning of its staff development effort for 
teachers and anticipates that its content will be modified /expanded as needed. 

Evaluation of Staff Development 

Evaluation of the administrators' staff development experiences (Manatt 
and work with the consultant) has primarily been conducted through informal 
small group debrief ings (likes, dislikes, etc) following training. The 
district has employed post-training questionnaires to assess teachers' red- 
actions (what they liked, would change and what they would like next) to the 
ProCEED training. As was noted above, participants' reactions to the ProCEED 
training and follow-up have been extremely favorable. 




Staff Development Schedule/Timeline 







Approximate 








Ifumber 


Training 


School Year 


Staff Trained 


Trained 


Format 


Summer 1981 i 


District adminis- 


15--20 


Trained by own staff 


'82, '83 


trators and super- 




experienced with the 




visors trained in 




Manatt process 




Dick Manatt super- 




(3-4 one-half day 




vision/evaluation 




sessions each summer) 




procedure 






Sunaner 1984 


District adminis- 


15-20 


Trained by consultant 




trators and super- 




(5 full days) 




visors trained in 








management style. 








learning style and 








Hunter-related 








instructional 








techniques 






1983-84 


Began training 


15-16 


Six full days of 




school staff in 6 


Teachers 


training by district 




project ProCEED 


(K-12) 


trainers 




instructional 


& 


(1 day per week over 




modules 


3-4 


6 weeks) 




(Group 1) 


Administrators 




1984-85 


Project ProCEED 


48^50 


Six full days of 




training of 


Teachers 


training by district 




teachers and 


(K-12) 


trainers 




administrators 


& 


(1 day approximately 




continued 


9-12 


everv 2 weeks ^ 




(Groups 2, 3 & 4) 


Administrators 




1985-86 


Project ProCEED 


Remainder 


(See above) 


1986-87 


training scheduled 


of school 


(1 day approximately 


and 


to continue 


staff K-12 


every 3 weeks) 


beyond 


(3 groups per year) 
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Implementation of District Supervision /Evaluation System 



Philosophy /Policy 

It is the district's philosophy /policy that teacher supervision and 
evaluation are a continuous and ongoing process. Supervision Is perceived 
as — "the process of a helping relationship In action, A trained observer, 
knowledgeable about the elements of the teaching act and the process of class^ 
room performance, observes and coaches the teacher about ways and means of 
self -Improvement, Evaluation Is the summative conclusion or staff rating 
reached after a series of lesson observations," The district generally sees 
teacher supervision /evaluation as being related or running together. Staff 
needing help, need to be Identified In the supervision process. The district 
plans to connect the elements/ content of ^supervision and evaluation In a more 
systematic oianner over the next few years. 

Requirements 

The district's requirements for supervision and evaluation are presented 
best In the descriptions of the district's observation and rating procedures 
included In the Tips for Teachers (1981) district publication. These are 
cited as follows. 

Observation . All tenured teachers shall be observed at least 
twice per year, once in each semester. All non-tenured 
teachers shall be observed at least four times per year, twice 
in each semester. Teachers new to the district shall also be 
observed at least four tlnies per year, twice in each semester, 
for the first full year of employment. Full-time substitutes 
will be observed, although they are not rated. Observations 
shall be made by the following personnel: (1) On the secondary 
level, at least one observation per year shall be done by the 
appropriate coordinator. At least one observation per year 
shall be done by the principal; and (2) On the elementary 
level, the principal/assistant principal shall complete the 
required observations. At least one observation per year 
shall be done by the principal. The appropriate curriculum 
coordinator may be called in for additional observations at 
the request of the principal. 

At least one observation per year will have a &chedu.1ed 
pre-obsecvatlon conference • The purpose is to assist the 
teacher and observer in defining the Important elements of the 
lesson, and to allow the observer to focus on those elements 
during the lesson. The focus on particular elements, however, 
does not exclude other observations concerning general 
teaching practices. Each observation shall last for at least 
one complete lesson. Secondary observations shall last one 
period. Post-observation conferences shall be conducted as 
soon as possible following every formal observation. The post- 
observation conference should be held between the teacher and 
observer no later than three working days after the obser- 
vation, 
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The written observation report shall be received by the 
teacher at the conference or not later than five working days 
after the observation. A copy will be given to the teacher, a 
copy kept by the principal, a copy sent to the superintendent 
for the employee's personnel file, and a copy to the appro- 
priate curriculum coordinator if applicable. An employee 
always has the right to respond In writing to any observation 
within five working days after receipt of the written obser- 
vation report* The response shall be attached to all copies 
of the observation form. 

Rating/Evaluation . All tenured teachers shall be rated 
at least once per year* All non- tenured teachers shall be 
rated at least twice a year, once in each semester. Full-time 
substitutes are not rated* The rating shall be completed by 
the principal, as the designee of the superintendent. The 
rating shall be completed on the East Penn School District 
alternative rating form, approved by the Pennsylvania Depart- 
ment of Education. The rating form must be completed within 
five working days of the last observation. The conference 
between the principal and the teacher must also be held within 
five working days of the last observation. The teacher will 
at that time be asked to sign his/her rating form. If the 
employee refuses to sign In the space provided, such refusal 
shall be recorded and dated. The employee will be notified In 
writing of this notation within ten calendar days. 

Final ratings shall be based on formal observations and 
the rater's judgment of the employee's total performance during 
the contracted teacher day. All teachers are encouraged to 
maintain a file which will aid the teacher and rater In arriving 
at a fair evaluation. The descriptors on the rating form are 
Intended to describe a good teacher in the East Penn School 
District, to describe the reasonable expectation that any 
teacher will usually exhibit these characteristics. Each 
descriptor will be checked as Satisfactory, Needs Improvement, 
Unsatisfactory or Not Applicable. Checks of Needs Improvement 
or Unsatisfactory require supporting statements* Each of the 
four categories will be summarized as Satisfactory or Unsatis- 
• factory only. The overall rating for the period shall be 
Satisfactory or Unsatisfactory. It Is possible that a gross 
deficiency in a single category might be sufficiently serious 
to warrant a total rating of Unsatisfactory, 

An overall Unsatisfactory rating means that a teacher has 
been Informed of unsatisfactory performance In one or more 
categories of professional performance. A rating of Unsatis- 
factory Is applied when the teacher's performance almost never 
exhibits the established criteria. This requires documen- 
tation. The rater will also provide written recommendations 
for Improvement and correction of any deficiencies. Two con- 
secutive Unsatisfactory ratings of a professional employee 
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shall be necessary to dismiss on the grounds of incompetency* 
The supetintendent shall sign all ratings. Unsatisfactory 
ratings shall be signed within five working days of the last 
observation, (pp. 57-59) 

An interesting feature of the district's observation process is the 
requirement that at least one observation per year involve a preconf erence 
with the teacher. The district has established guidelines for the pre- 
observation coiiference. The one page guidelines form directs the observer 
(principal/supervisor) tot (I) discuss whete the teacher is in the course 
(unit, lesson, page, etc.); (2) inquire as to the objectives of the lesson 
observed; (3) determine what teaching/learning activities the teacher plans 
to use; (r) inquire upon what aspects of the lesson the teacher would like 
the observer to monitor (the following ideas are examples: statement of 
objectives, attention to reviews task-orientation of lesson, clear tran- 
sition signals, verbal or non-verbal emphasis, clarity of instructions, 
student comprehension, evidence of teacher preparation, and attention to 
summariaa::ion); (5) determine how the teacher plans to evaluate the achieve- 
ment of the objectives; and (6) ask if there ate any group or individual 
characteristics or circumstances of which the evaluator should be aware 
(unusual behavior, group interactions, students leaving class during the 
period, lab work). 

Organization and Process 

The district's teacher supervision evaluation process is carried out by 
the school principals, assistant principals and curriculum supervisors. On 
the average, elementary school principals and their assistants each super-- 
vise/observe 30 teachers, secondary principals — 40 teachers > and curriculum 
supervisors — 30 to 40 teachers. Principals and their assistants are ex- 
pected to spend approximately 30% of their time on supervision/observation 
and related instructional activities, while supervisors are expected to 
spend 30-35% of their time on the same. There is an attempt at the secon- 
dary level to assign administrative staff, for observation purposes, to de- 
partments or areas in which they have some content expertise or background. 
The superintendent also does some teacher observations (5 this past year) as 
does the director of curriculum and instruction (17 this past year). 

The observations of teachers focus on the various elements of instruc- 
tion, organizational patterns in the class, teacher planning and questioning 
techniques. Observers look for effective lesson introduction, development, 
and closure. Writeups of observations include brief lesson descriptions, 
commendations, recommendations for Improvements, and a summary. Both 
announced and unannounced observations are made» As noted earlier, one 
observation per year involves a structured p re-conference and all obser- 
vations involve post-conferences. Several of the administrators have 
shared with their staff, lists of the elements of effective lesson design to 
give teachers an Idea of what they will be looking for when they observe. 
There is some variability among the lists, as different administrators have 
pulled together or synthesized different aspects of the content of the 
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training they have received related to instruction. The lists vary in 
content, number o£ items, and format. In most cases, however, the teachers 
have not yet received formal training in the content /skills addressed in 
these lists. 

System Monitoring 

The superintendent monitors the implementation of the supervision/ 
evaluation system primarily by reviewing all the formal observations and 
evaluations, submitted by his staff, for consistency and accuracy. The 
superintendent also interacts formally with his staff three times per year 
(August /September , mid-year, and May) to set beginning-of-the-year admini- 
strative goals, check on progress, and engage in end-of-the-year self- 
evaluation, respectively. Supervision/evaluation is part of the staffs' 
MBOe, and 30% of their evaluation relates to their supervisory /evaluation 
duties. 



Administrators' Perceptions 

Two assistant high school principals, an assistant junior high school 
principal, two elementary school principals and one assistant principal, and 
two curriculum supervisors (K-12) were interviewed for their perceptions of 
the district's teacher supervision/evaluation system. 

Relationship Between Teacher Supervision and Teacher Evaluation 

All administrators reported that the district was trying to make a dis- 
tinction between teacher supervision and teacher evaluation. Only one saw 
it as a very clear distinction. Most saw the procedures as related. Most 
also reported that the purpose of the supervisory observations was to help 
teachers improve. One administrator took exception with the word "improve" 
and said supervision was more of a monitoring process of what was going on, 
and that part of the function was to "highlight what is going well." 
Another stated that, "In theory » the purpose of supervision is to help 
teachers improve, while evaluation is to rate them — however, this is not 
always the case in district practice." 

Utility of Staff Development 

Overall, the administrators spoke positively of the training (Manatt, 
Harvey Silver, ProCEED) that they had received. They liked the analysis of 
the Manatt videotapes and various aspects of the consultant's training 
(e.g., the emphasis on active student participation and ways to promote it, 
the questioning techniques, the one-on-one interviewing techniques, the 
hands-on experience and role playing, and the "modeling of what good instruc- 
tion is"). Most indicated they had modified their behavior due to the train- 
ing. One said this was the first district he had ever been in where he was 
trained. Almost all indicated that they were receptive to more training. 
Two indicated that they "would like more training in some of the things 
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silver didn't get to; skills in conferencing teachers." One said, "iM like 
someone to teach me how to assist a teacher who is fearful of change (the 
TS/E process) .how to reach out to those people." One also indicated there 
was some concern and /or resistance among staff regarding the Silver 
training. 

Feasibility of Impleipentation 

Most administrative staff indicated that they spend a fair amount of 
their time on supervision and evaluation. The estimated percent of time 
spent varied. The figures cited by the eight administratative staff inter- 
viewed were: 10, 15, 20, 30, 30, 33, 35, and 50 percent of their time, 
respectively. They all indicated that the clinical observation involving 
the pre-conference (30-45 minutes) took more time overall (two to two-and- 
one-half hours) . Most indicated that they have shared some of their train- 
ing materials (handouts on effective teaching steps) with staff. Several 
felt that different observers may be emphasizing different things (teaching 
components) to varying degrees. Several also indicated that some principals 
had modified the district's pre-conf erence guidelines. The supervisors in- 
dicated they had 55'-60 staff to supervise; elementary principals had 20-40 
staff; and secondary administrators had some 60 staff to supervise. 

All staff indicated ways in which they worked with teachers; providing 
suggestions, resources, and in a few cases doing some model teaching. 
Several emphasized that they tried to give teachers positive reinforcement. 
A variety of data collection modes were reported. Some staff tried to take 
verbatim notes during observations. Others noted comments at various time 
Intervals, One took brief notes on cards. 

Impact of Supervision 

Staff shared mixed perceptions of the impact of supervision. About a 
half thought it definitely had an impact and helped teachers improve. The 
remainder expressed some uncertainty of how effective it was with veteran 
teachers (*'lt really depends on the teacher's attitude. Some are closed 
out,"). Several felt they had more success with newer teachers whom they 
generally reported spending more time with. The curriculum supervisors 
also related some problems in interacting with elementary staff ("They 
didn't know us and were unsure of what to expect,"). One administrator was 
not sure if the system really helped staff improve. He felt it needed to be 
linked in a meaningful way to the staff development program for teachers. 

Administrators also varied in their perceptions of how teachers have 
reacted to the district's supervision system. Two or three administrators 
perceived that teachers would like more closure on the district's expec- 
tations ("They are not really sure wh..t we are looking for given our new 
training."; "I think teachers would like more follow-up to get the big 
picture.'^). Most seemed to feel, however, that there was increased accept- 
ance by teachers of the teacher supervision/ evaluation system. One, 
however, felt that experienced teachers perceived the system and related 
procedures as a form of harassment. 
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Impact of Evaluation 

Most administrators felt that the district had made some strides in 
improving the evaluation procedure. They felt it had helped them to get to 
know teachers better. They perceived that staff saw evaluation as a 
"necessary process," No strong comments were made, per se, about the degree 
to which it helped teachers improve. A few principals, however, reported 
that more work might be done with marginal teachers — ^more follow-up. All 
noted that there was adequate room for written commendations both on the 
supervision and the evaluation forms. At least two also commented that they 
would like to see the formal rating scale expanded on the positive side to 
include a category, other than "satisfactory," for above average teachers. 

Perceived Strengths 

The most commonly perceived strength, mentioned by half of the admin- 
istrators, was that in comparison to past procedures, some standardization 
had been introduced into the teacher supervision/observation process. Two 
staff also felt that teachers had a more positive attitude toward the 
program. Other strengths, each cited by a different administrator, are 
listed as follows. 

• "Just the right amount of formal observation has been established. 
If we want to go beyond it, we can. In addition, evaluation is now 
done personally (face-to-face) and not by a letter," 

• "The system provides teachers with positive feedback and 
suggestions." 

• "Administrators are more sensitive to teachers and the complexity of 
teaching." 

• "The process is in writing — teachers know what to expect," 

• "Each teacher is observed by more than one person. Each teacher 
also gets a walk- in and a scheduled clinical observation," 

Perceived Problems 

Half of the administrators reported that finding time to conduct the 
observations and conferences was a problem. Two staff also mentioned that 
they had difficulties in providing constructive criticism/ recommendations to 
teachers ("They see it as negative,"). Other problems, each mentioned by at 
least one administrator, included the following. 

• "Teachers are not sure what we are looking for given our new 
training." 

• "Scheduling post-conferences is a problem," 

• "The pre-conference form is not being implemented consistently. Some 
administrators have adapted it." 
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• "Some teachers still view the process as a *gotcha game.*" 

• "Working with short-time teachers is a problem. They think they are 
beyond any need for improvement," 

• "The criteria for giving 'Needs Improvement' evaluation ratings are 
unclear." 

• "The required observations have Qut down on the time building prin- , 
cipals have for casual , informal drop-ins to teachers' classrooms," 

• "It's difficult to play both a helping role and an evaluative role, 
I'd rather not be the heavy but I'm in management," 

Perceived Needs 

Tvo administrators mentioned that there was a need to provide inservice 
to teachers in the "steps in the teaching act" ("Teacher inservice needs to be 
coordinated with teacher supervision/evaluation/')* Two also mentioned that 
there was a need for more time to work with teachers by following up on the 
observations. In addition, each of the following recommendations were made by 
at least one administrator* 

• "Change the evaluation rating system to include a category beyond 
satisfactory, to afford recognition to superior teachers," 

• "Increase the time between observations and required feedback 
(post-conferences) to 10 days, to facilitate scheduling," 

• "Provide administrators time to work with outstanding teachers so 
administrators can learn from them*" 

• "Get more scientific and systematic about the teacher supervision/ 
evaluation system*" 

• "Spend more time helping teachers who need to improve their per- 
formance," 



Teachers' Perceptions 

Forty- three teachers were interviewed to obtain their perceptions of the 
district's teacher supervision/ evaluation system. Nineteen taught at the ele- 
mentcry level, 9 at the junior high level, and 15 at the high school level. 

Teacher Supervision and Teacher Evaluation 

Most teachers saw supervision and evaluation as being relat;ed ("When- 
ever they observe us they are evaluating us,"). Most staff expressed the view 
that the current observation/evaluation process was a definite improvement 
over the evaluation procedures used several years ago ("The mechanics of it 
are more structured*" "It's nice to be cozmended*" "The conferencing is a 
good feature*" "They're attempting to be more consistent," "They are trying 
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to help us grow."). Regarding the implementation of the system, the teachers 
generally reported experiencing the same process (drop-in and clinical obser- 
vations) as described by the administrators. 

Staff had a less clear understanding, however, of tbe training adminis- 
trators were receiving; and of how the district planned to relate Project 
ProCEED to teach observation/evaluation. There was concern as to how it would 
all fit together. Three or four staff indicated they would have liked more 
in-depth training or exposure to the new lesson format terms that the adminis- 
trators were using. They also expressed uncertainty over whether the sugges- 
tions administrators were making regarding these terms were "just suggestions" 
or whether they were "the district's policy for observation criteria." 

Satisfaction With Input 

Most staff who experienced Project ProCEED training reported their satis- 
faction with the training and their pleasure and pride over the fact that such 
a quality program had been developed and presented by district staff. Staff 
were less certain about the content and purpose of the the training admini** 
strative staff had befin receiving related to teacher supervision/evaluation. 

Utility/Impact of Supervision/Evaluation and Project ProCEED 

There were mixed perceptions of the utility of the recommendations made 
as a result of the observations, especially among the more experienced staff. 
The great majority of the staff saw them as pro forma in nature. A lesser 
number of staff reported that the recommendations had some utility to them — 
and attributed che positivity of the experience to their "particular" princi- 
pal's skills, actitude, and personality. Those teachers relatively new to the 
district who had been through ProCEED, generally spoke more positively about 
the utility of the observation process. The newer teachers also spoke highly 
of the utility of the district's induction process^ There also appeared to be 
about equal preference expressed for "drop-in" versus "clinical" observations. 
Reading between the lines, those staff who appeared to take the process more 
seriously, expressed a stronger preference for clinical observations (more 
opportunity for communication, clarification, etc^)* 

Twenty- two teacher's who had experienced Project ProCEED were also asked, 
"What percent of the total knowledge and skills that you currently employ in 
teaching (independent of content knowledge) would you attribute to: (1) your 
practice teaching experience; (2) trial-and-error learning on your own over 
the years and/or interaction with peers; and (3) your Project ProCEED /Super- 
vision experiences?" The majority of the teachers attributed more than half 
of their teachir^g knowledge/ skills to their own trial-and-error experiences 
and to their interaction vich peers. The teachers attributed the following 
percents of their teaching knowledge and skills to their district ProCEED/ 
supervision experiences: 45% attributed 20% to 25% of their skills to the 
district's recent program; 18% attributed 10% to 15% to the program; 14% 
attributed between 5% and 10% to the program; and 23% attributed relatively 
little of their knowledge and skills to the program (0% to 2%). 

Consistent with the findings reported above, most all staff who had 
participated in Project ProCEED spoke positively of the experience* Several 
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indicated that they had changed one or more of their teaching behaviors (e,g», 
quescioniugi wait-tlmc» THSA> Btatement of objectives i etc.) because of it* 
Many thought it reinforced things they already did, was useful as a review, 
and made them more aware of what they were doing in the classroom. Many are 
looking forward to more staff development and would even support it being tied 
to class observations, if they were non-threatening, long-term and develop- 
mental « 

Utility/Impact of Evaluation 

To a large extent, staff were indifferent to the end-of-year staff rat- 
ing* They viewed it neither positively nor negatively, it is just there* It 
was a pro forma exercise* Most felt it played little or no role in helping 
them improve their performance* Several noted > however* that it would 
definitely attract their attention if they received a "Needs Improvement" or 
"Unsatisfactory" rating* 

System Fairness 

Staff expressed mixed reactions i some in favor and some not» to the idea 
of putting more teeth into the staff rating form* Some would like a wider 
range of categories to recognize outstanding service; and identify teachers 
who are not carrying their load* Those expressing opposition based it, in 
part, on whether or not administrators had the skills to make these decisions, 
and on the criteria to be used. Several felt that both good teaches and mar- 
ginal teachers receive the same "Satisfactory" rating* Several also felt that 
more narrative praise could be used on the staff rating form ("We write com- 
ments on our report cards; why can't administrators do the same with us?") 



Staff appeared to share a common and positive perception that the dis- 
trict was moving to create a program to support the professional development 
of teachers* Perhaps a third of the staff expressed a desire to be treated as 
professionals, and saw Project ProCEED as a step in that direction* They want 
more humanism ir the system and appreciate the two-way communication afforded 
by the clinical process* They saw the clinical process as being more compat- 
ible with the goals of ProCEED than the drop-in process. They also perceived 
some potential professional benefit in videotaping and peer observation » if 
dealt with in a totally non-threatening context by the administration* 

Needs /Problems 

There were perceptions by at least a third to one-half of the staff that 
there was variability among and between schools (administrators and super- 
visors) , and years with regard to the focus and rationale of the observations. 
Staff made the following comments. 

• "Last year they focused on 'A', this year 'B' » and next year who 
knows what it will be/' 

• "Also* some principals are using ProCEED terminology with teachers who 
have not had the program." 



Strengths 
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• "This year their expectations are clearer than in the past." 

• "Administrator A is more negative than administrator B." 

• "They have to make some recommendations for improvements even if they 
are trivial and at times even conflicting/* 

• "Curriculum supervisors tend to be more critical than principals." 

Ir. actuality, most teachers did not have an accurate perception of the 
application and/or results of teacher evaluation across the districts It 
appeared to be due to a communication problem or issue. 

A few staff also noted that the district's policy regarding the inter-- 
pretation of the "Needed Improvement" (NI) category of the Professional Staff 
gating form needed clarification. They perceived that there was uncertainty 
among administrators regarding when to use the NI category, and that there 
were inconsistencies in the criteria used to arrive at NI ratings* They added 
that the administration had stated that a NX rating was not pejorative, per 
se. They felt, however, that most teachers viewed NI ratings quite nega- 
tively. In addition, when asked to cite the most important criteria on which 
they were evaluated, staff gave a variety of responses • Although all re- 
sponses were relevant (e.g., preparation, lesson flow, discipline, etc«)t 
it was apparent that staff did not share cotnmon expectations and a common 
language regarding the observation/evaluation process* 

Finally, a few changes were suggested regarding the observation pro- 
cess * 

• "iM like to receive the written observation report before the 
meeting." 

• "Have more frequent informal observations with no official write-up." 

• *'To be informed of results during non-class time." 

• "Observe earlier in the year*" 

• "Need to recognize strengths more than they do." 

Five or six staff also indicated they would like to see administrators 
and supervisors teach some lessons occasionally. Particular reference was 
made regarding supervisors who apparently have turned down some teachers* re- 
quests to put on a demonstration lesson to "show-me-how." It was stated that 
compliance with the above suggestion would aid communications, enhance credi- 
bility, and make staff more receptive to evaluators' comments and recommen- 
dations. 
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District Administrative Staff's Overall Reflections on 



Program Design, Operation, and Impact 



Perceived Strengths 

When asked about the strengths of the program, the superintendent and the 
director of curriculum and instruction cited the following: 

• "The teacher supervision/evaluation system has helped a number of 
teachers to improve." 

• "The staff development program has increased principals' and super- 
visors* knowledge of the components Ipvolved in quality instruction. 
Administrators have moved from understanding to application of this 
knowledge in their teacher observations and in-depth conferences." 

• "Teachers are aware that the district's number one priority is to 
improve instruction through a 'helping process' as opposed to a 
'gotcha mentality*'" 

• "Project ProCEED has helped us the most. It has been well received 
and has built knowledge and trust* It is an Investment in people as 
professionals/' 

Perceived Problems 

The major problems reported by central office staff were: supervision/ 
observation procedures were not sufficiently standardized across schools; 
there had been a few administrators who had showed favoritism or bias in their 
supervisory duties; the category descriptions on the DEBE-333 were ambiguous 
and not fully understood; the role of the supervisors was not fully defined at 
the elementary level; there was a failure on the district's part to communi- 
cate fully and clearly their expectations regarding. teaching to the teaching 
staff; overcoming the resistance of some administrators about going into the 
classroom to do in^-depth observations/conferences; teacher's concerns that, if 
they volunteered for ProCEED training, they might be held accountable via the 
observation-process for demonstrating all the ProCEED skills covered. In 
fact, central office staff said, "We did tell teachers that if they went 
through ProCEED training, they would be held accountable (for demonstrating 
new skills) through the clinical process — however, we have not clarified how 
and whan." 

Changes Made or Things They Might Do Differently 

There are a number of changes that central office staff anticipate making 
in the various components of the teacher supervision/evaluation program* They 
plan to: (1) schedule three weeks between each Project ProCEED input session 
in 1985-86 to afford more time for teacher application and feedback; 
(2) synthesize/integrate and coordinate all the elements of the district's 
program (administrator's staff development, the consultant's input. Tips for 
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Teachers, Project ProCEED); (3) clarify the district's expectations regarding 
teaching, supervisory observations, and how these relate to the various pro- 
grams; (4) clearly communicate these expectations to teachers — make them 
official; (5) provide time for dialogue with staff to build understanding 
and trust; (6) increase the standardization of administrators' supervision/ 
evaluation activities; (7) spend more time working with teachers most in 
need of assistance; and (8) provide for some coaching for administrators to 
increase further their skills in observing and conferring. 

When asked what they would do differently if they had the chance to do it 
all over again, central office staff indicated they would: (1) establish the 
elements of a teaching model or effective lesson up front, so staff would have 
a coimnoa understanding and common expectations; (2) address the elements of 
effective teaching in the context of a single model in Project ProCEED; and 
(3) work with the consultant more closely over a longer period of time and 
come to closure on specific goals. They perceived that they were, in some 
vays, reactive to the consultant. The second time around, they plan to have 
clearer expectations and goals regarding the outcomes they expect from working 
with the consultant. They plan to direct and shape his input to achieve their 
vision of their goals. 

Assistance They Could Have Used 

Administrative staff reported that they could have used some assistance 
in tutoring or coaching principals and supervisors in their clinical roles — 
to work toward both greater skills in observing/conferring and increased con- 
sistency.' They perceived that there was little difference between the 
clinical and the drop-in teacher observation write-ups or reports. 

Advice to Others 

When asked what advice they had for other districts considering the 
development of similar teacher supervision programs, they stressed the 
following. 

• Work with the union to build trust. Develop an improvement 
oriented program designed to help all staff. 

• Train staff K-12 in small groups (20-25). Involve administrative and 
teaching staff as co-presenters to build collegiality and 

trust, 

• Give staff time to practice the skills in which they are being 
trained. 

• Do not underestimate the importance of communications. Make sure all 
staff get the message. 

• Involve all staff in program development /implementation decisions. 
Work toward staff commitment to the process. There is a difference 
between compliance and commitment. Commitment is the desired end. 

• Have a clear vision or picture of what the program will look like at 
the end. 
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Summary of Findings and Discussion of Implications/Issues 



Summary of Findings 

The East Penn School District Is moving forward on a number of fronts to 
Improve the quality of teacher supervision and evaluation, and the overall 
quality of Instruction being delivered by district teachers. The district has 
made definite strides In Its efforts to Improve administrators' capabilities 
In the areas of knowledge of effective Instruction, teacher observation, and 
post-conferencing procedures ♦ The district has attempted to Introduce a 
measure of standardization In the above procedures ♦ The district has also 
Instituted a staff development program to upgrade teachers* skills In six 
areas related to effective teaching* 

Some of the strengths of the system that appear to have contributed to 
the success achieved to date Include the following* 

• Summer training for administrators In the Dick Manatt model of 
supervision* 

• Training for administrators In several models of effective 
Instruction. 

• The adoption of a modified DEBE-333 supervision/evaluation model which 
Involves at least one clinical teacher observation per year (pre-con-- 
ference, observation, and post-conference)* 

• The development of a quality R&D-based training program for teachers 
(6 days) which Is conducted by qualified In-house staff (teachers and 
administrators) * 

Some of the problems experienced by the district which, to some extent, 
have placed a celling on the level of success achieved to date appear to be 
attributable to the following, 

• The district did not adopted a "single" model of Instruction (In toto 
or eclectic) as the official district model, per se* Consequently, 
although administrators and supervisors have attempted to Implement 
given supervision/evaluation procedures, they have tended to focus on 
different models and aspects of Instruction In the context of 

the district ^s system* 

• The district's training of administrators has not progressed as far as 
might be necessary /desirable with regard to follow-up on-site coaching 
of administrators as they engage In Implementation of the process* 

• The district has not yet tied together conceptually the various 
ongoing staff development programs with regard to their relation to 
teacher supervision/evaluation. This has resulted In mixed signals 
being received by staff regarding district expectations and district 
policy* 
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• Because of changes in leadership and due to the 'ariety of different 
topics in which administrators received training over the past five 
years, the district's vision of exactly where it wanted to go unfolded 
in a somewhat piecemeal fashion* 

A number of required elements are now in place. A fev more need to be 
addressed. District central office staff are fully aware of the above and 
are planning to take action to coordinate their efforts both conceptually 
and procedurally to achieve greater levels of program integration and in*- 
creased levels of staff competence, satisfaction, and connnitment . 

Implicat ions/ Issues 

The experiences of the East Penn School District are noteworthy from a 
number of perspectives that other districts might find profitable, 

• First, they attest to the fact that districts can mount focused staff 
development programs for teachers, using their own staff, if adequate 
resources are allocated for planning, development, and delivery over 
a reasonable period of time* 

• Second, the district's experiences attest to the desirability of 
having a clear vision of where a district wants to go with a given 
instructional improvement supervision program. East Penn's exper- 
iences highlight the desirability or, in fact, the necessity of 
defining how various programs relate to one another---clear district 
expectations of teachers and administrators need to be established If 
a supervision/ evaluation program is to be successful, 

• Third, the district's experiences might also be interpreted as sup- 
porting the conclusion that .success in changing school staffs' skills 
is best achieved if training in content and skills (for administrators 
or teachers) is divided into discrete segments with time and resources 
afforded for practice, coaching, and feedback on given skills in a 
serial or incremental order. If a district places too much content or 
too many skills before staff and spends only a moderate amount of time 
on each, or fails to define how they interrelate, significant change 
may not occur — in fact, noise and' mixed signals may be introduced into 
the system, 

• Finally, the data support the conclusion that teachers need to be 
trained, and given feedback on their performance over time, on "a 
model" of instruction. If administrators are trained and teachers 
are not, problems will ensue. 
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CASE STUDY 3: PITTSBURGH PUBLIC SCHOOLS 



Background Data 
Schools Students 

• 53 Elementary (K-5) • 40,257 students 

• 17 Middle (6-8) • 52% black 
© 15 High Schools ^ A7% white 

• 3 Special Education Centers • 1% hispanlc 

Staff 

• Superintendent 

• 6 Assistant Superintendents (also serve as department heads) 

• 91 Principals 

• '65 Assistant Principals 

• 64 Supervisors 

• 26 Division Heads 

• 977 Elementary School Teachers 

• 2,089 Middle, Secondary and Special School Teachers 

• 7.7Z of staff (N=237) laid-off in the last three years 
(79% of staff = 9+ years of experience) 

Average Per-Pup\l Expenditure • 

• (N/A) 

Teacher Supervision /Evaluation 

• Modified DEBE-333 procedure, instituted in 1985-86. 

• Hunter-based staff development initiative, Pittsburgh Research-based 
Instructional Supervisory Model (PRISM) initiated in 1981; currently 
comprised of four progranmatic efforts: PRISM I, II» III and IV. 

• Teacher evaluation has been included in the collective bargaining 
agreement with the Pittsburgh Federation of Teachers since 1967. 

Community 

The City of Pittsburgh School District, located in Allegheny County, 
servG3 some 425,000 residents. As a large urban area, the district serves 
a heterogeneous mix of students. Family backgrounds represent a mix of all 
professions and vocations from highly educated professionals to unskilled 
laborers and unemployed. The total population of the city decreased by 
96,000 persons or 18.5% during the ten-year period ] 970-1980. The school 
age population decreased by 39%, or 45,000 students, during that same 
ten-year period. At present, the area is slowly recovering from the 
economic privations brought about by the decline of the area's steel 
industry. 
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Program Initiation, Planning > and Start-up 



Initiation 

The Pittsburgh Research-based Instructional Supervisory Model (PRISM) ♦ 
and other Pittsburgh Public School educational Improvement Initiatives, were 
Initiated as a result of the leadership of the superintendent who assumed 
office In September i980. In a review paper, A Superintendent of Education; 
Opportunity for Excellence (Wallace, 1985) ,* the superintendent outlined his 
views of the Instructional leadership role of the superintendent, as It 
relates to Improving public education In today's schools, and chronicled the 
activities he Initiated In Pittsburgh to Improve education. These activities 
can be described as follows. 

• A distrlct-wlde needs assessment, conducted by Dr. William Cooley 
and his staff at the learning Research and Development Ceiiter 
(October 1980-January 1981) at the request of the superintendent, 
led to the school board's January 1981 adoption of school improve- 
ment and cost effective management as the priority goals for the 
district. Within these two major goal areas the board resolved, in 
Febri\arv 1981, to allocate resources and concentrate efforts on 
solving specific problems Identified In several areas. The problem 
areas listed In order of priority were: (1) students' basic skills 
achievement, (2) staff evaluation, (3) student discipline, 

(4) attracting and holding students, (5) enrollment decline, and 
(6) Increasing the effectiveness of specific schools. The district 
subsequently took coordinated action on all of the above, 

• The present study focused on the actions taken regarding the needs 
Identified In the second most Important priority area, staff 
evaluation,** The needs assessment revealed that administrators, 
teachers, and supervisors perceived a lack of clear criteria for 
Identifying effective teaching strategies and a lack of common 
standards for conducting teacher performance evaluations. There was 
general agreement that current evaluation practices needed revision. 
The seven most frequent suggested revisions were: (1) the 
development of mutually agreed upon goals that reflect both system 
and job objectives; (2) more time for direct contact/observation 
between the reviewer and the person being reviewed; (3) a style of 
personnel review that is more formative (I.e., support for improve- 
ment) than summative in nature; (4) build incentive systems that 
encourage and recognize quality of performance; (5) follow-through, 



*EKS Management Operations Information Exchange , Summer 1985. 

**The first 31 pages of this report (Targelj descriptive) were based 
on: (1) information acquired during the interviews of central office and 
staff development team members; and (2) Information gleaned from some AO-50 
Internal PRISM documents and publications obtained from district staff 
development team members and other school staff. 
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first for improvement > but failing progress^ for accountability purpose 
(6) broaden the participatory base and direction of personnel reviews, 
permitting self-assessment information to be included and encouraging 
input from staff on the performance of superiors; and (7) improve the 
current training/capability of those charged with the responsibility 
for reviewing personnel. 

Planning and Start-up 

In March 1981, the superintendent convened an Instructional Leadership 
Committee. The 20-member committee of principals, teachers, supervisors, 
and central office staff was charged with developing a unified plan for 
improving instruction, instructional leadership, and personnel evaluation 
in the district. The superintendent saw these three areas as being closely 
related and directed the committee to incorporate an instructional 
leadership model in the plan to serve as the foundation for improving both 
instruction and evaluation. In the course of reviewing the literature and 
discussing the task (March-June 1981), the committee reached a general 
consensus that the model chosen should meet the following research-related 
criteria. It should: (1) support the role of the teacher as a decision 
maker; (2) integrate or support the concepts of mastery learning theory; 
(3) address time-on-task; (4) be based on reinforcement theory; (5) incor- 
porate student motivation theory; (6) be adaptable to a clinical mode of 
supervision; and (7) be research-based, practical and realistic. The 
committee also defined instructional leadership as "a process of actively 
influencing others to establish and use mutually agreed upon methods to 
achieve desired educational goals. An instructional leader is able to act 
in a manner which influences and teaches others how to bring about the most 
effective learning in pupils. This necessitates appropriate knowledge, 
attitudes, and skills." 

After considering numerous models the committee decided that Made- 
line Hunter's Model of Effective Teaching best met the above criteria. In 
addition, in the committee's judgment, the Hunter model demonstrated the 
greatest flexibility and adaptability to all grade levels and all content 
areas. After reviewing the implementation efforts of other school dis- 
tricts using the Hunter model, the committee also decided to use Ted Forte, 
Director of Staff Development and Human Relations for the Norfolk, Virginia 
School District, as a project consultant. Following a Workshop on Instruct* 
tional Skills (overview of Norfolk's staff development model) for the 
committee and an overview presentation for district administrators by Forte 
the committee recommended that Pittsburgh adapt the Norfolk, Virginia 
School District's staff development-based instructional improvement and 
clinical supervision model to the needs of Pittsburgh. 

The nev; program was called the Pittsburgh Research-Based Instructional 
Supervisory Model (PRISM). The primary goals of PRISM were to foster 
effective teaching through staff development programs designed to upgrade 
the observation and evaluation skills of principals, establish a common 
language of Instruction and a set of standards for personnel evaluation, 
and provide clinical teaching experiences for both principals and teachers. 
A five-year timeline was proposed for the program with the intent that 
PRISM would become the standard for the formal evaluation of teacher 
performance once it was fully disseminated district-wide. 
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In June 1981, the Pittsburgh board of dlrecccrs formally endorsed the 
recommendations of the Instructional Leadership Couunittee and authorized 
Che formation oE a staff development team (SDT> to plan, develop, and 
conduct PRISM training; monitor its implementation in the schools; and 
establish procedures for evaluating the program* The SDT consisted of a 
director (from central office) and four* associate directors or team members 
who were former administrators with extensive backgrounds in instructional 
supervision, curriculum development > and admin is t rat ion ♦ Four of the team 
members had served on the Instructional Leadership Committee. The initial 
goals of the st.iff development team were to: (I) acquire the required 
theory by learning the Hunter model and modeling continuously all aspects of 
the theory; (2) develop training skills in teaching the content of the model 
through practice with each other and with the consultant; (3) establish 
credibility within the district by making simulation materials and films, 
and teaching students in Pittsburgh schools; and (4) design a system for 
staff development in Pittsburgh schools by organizing a permanent training 
center, designing training materials, and developing a process of 
involvement for administrators* 

During July, August, and September 1981; the SDT addressed the above 
goals and related activities. They attended training sessions conducted by 
Ted Forte in Pittsburgh and in Indianapolis, and by Madeline Hunter in 
Boston. In Indianapolis they observed Forte training school district staff. 
They worked intensively both individually and as a team to: (1) develop 
the workshops for principals and teachers, based on their own learnings of 
the Hunter Effective Teaching Model, that defined the evolving PRISM 
program; and (2) design the system or procedure for its incorporation into 
the Pittsburgh school system. The team incorporated the elements of a 
R&D-based staff development model into the design of PRISM. The elements 
consisted of presentation of theory, demonstration or modeling of the 
skills being taught, guided practice by participants in simulated situa- 
tions, provision of structured feedback regarding the practice, and follow^ 
up on-site coaching of the participants during the actual application of the 
skills. 

Overall, SDT members used several consultants in the first two years to 
assist with program conceptualization and/or aspects of training ♦ Ted Forte 
was used for five days in the summer of 1981 to assist with planning. Four- 
teen months after the program was initiated, Ernie Stachowski, of Long Beach, 
California, was brought in for a day (October 1982) to assist with planning/ 
training. Madeline Hunter was also brought in for a day at the end of the 
second year of the program to address a large group of school staff* 

The long term role of the SDT Involved the primary responsibility for 
conducting the inservice training and follow-up sessions for all district 
administrators involved with instruction • They were also assigned the 
addtional responsibilities of ongoing inservice related to instruction and 
PRISM implementation monitoring and evaluation* Overall, the program was 



*Currently, there are five accociate directors or team members. 
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viewed as dynamic and evolving from Hunter's framework within the con- 
straints of specific district goals. In that regard, the SDT monitored and 
adjusted their own activities and Job descriptions in light of the project 
goals and resources. It was eventually decided that each SDT member would 
teach the adapted Hunter model and provide structured feedback to partici- 
pants in the form of coaching and conferencing. The SDT also developed five 
coordinator roles to be split among team members: (I) the director would 
continue to coordinate the overall administration of the team, (2) the co- 
ordinator of research and evaluation would establish and implement an eval- 
uation and monitoring system to assess and track district progress, (3) the 
coordinator of system-^ide implementation would institute and maintain a 
system of follow-up resource support, (4) the coordinator of inservice 
training would develop and implement an action plan for related inservice 
programs, and (5) the coordinator of administrative growth and development 
would prepare and carry out programs to broaden the scope of educational 
leadership training. Each of the roles involved, to varying degrees, re- 
lated materials preparation, liaison, and planning activities. A more de- 
tailed description of SDT roles/activities is presented in this report in 
the section. Implementation of District Supervision/Evaluation System, 
Organization and Process* 

To a large extent, systematic program planning and needs sensing have 
continued throughout the duration of the project as SDT staff acquired in- 
creased experience, identified additional needs, conceived of ways to ad- 
dress those needs, and affected the required modifications and/or additions 
to the program* This systematic opportunistic approach to educational im- 
provement was built into the design and organizational structure of the pro- 
ject. The project was conceived by the district as being dynamic in nature, 
and SDT staff and other core group staff have met with the superintendent on 
a regular basis since the inception of PRISM to expedite communications and 
decision-making. In that regard, by the summer of 1982, PRISM evolved into 
three separate but related program efforts, PRISM I, II, and III. Two years 
later, PRISM IV evolved. 

In the vords of the superintendent, "PRISM I is concerned with provid- 
ing a consistent framework for the improvement and evaluation of instruction 
at all levels in the district. PRISM II is directed toward improving the 
instructional leadership of principals, supervisors and central office admin- 
istrators. PRISM III is the district's effort to improve the quality of 
secondary education. PRISM IV is the districts effort to improve the 
quality of elementary education. All four PRISM programs are designed to 
improve the effectiveness of instruction and thus lead to a higher quality 
of student learning in the district." 

The specific charge to the SDT, during program start-up in the 1981-82 
school year was to: 

• provide inservice training in the use of the Hunter model, at the 
district training center, for central office staff, principals, 
selected supervisors, and selected assistants 

• conduct the inservice training in a manner that emphasized the role 
of the principal as the instructional leader 




• work with the principals during training to improve skills in the 
use of the models classroom observacion, and conferencing 

• maintain records to monitor the imple^ientation of the program in 
each building 

• provide instructional materials for principals to use 

• advise principals in the inservice training of staff 

• provide follow-up assistance to principals in classroom observations 
by attending observations and post-observation conferences and 
discussing both with the principal* 

Program introduction consisted of a one-day inservice session conducted 
on five separate dates in September-October 1981 by Ted ¥orte and the SDT 
staff to provide an overview of the PRISM goals, content and timeline for 
all administrators, the board, the Pittsburgh Federation of teachers and PAA 
representatives. Administrators of like role assignments were assigned to 
each of the meetings* 



Program Funding 

Annual Costs 

The PRISM program has been included as a line item in the district 
budget since its inception in 198K The approximate costs per year have 
been: 

Start-up 1981 $125,000 Salaries 

$ 5,000 Consultants 
Continuation 1982 $352,797 

1983 $684,939 

1984 $508,373 

1985 $532,449 



Allocation of Monies 

In the start-up year, the majority of the PRISM program expenses were 
associated with the salaries for staff development team personnel and the 
clerical support staff involved in the effort* The salaries for the staff 
development team and related clerical support personnel have continued to 
comprise 30% to 40% of the annual cost of the program. The other program 
expense that has absorbed the bulk of the remaining annual program cost has 
been the expenditure of monies for teacher inservice salaries* These monies 
have been spent on salaries for the training of Schenley High Teacher Center 
resident and replacement teachers both during the summer and after school 
hours (PRISM III), Expenditures for consultants, materials and other 
miscellaneous costs have typically constituted less than two or three per- 
cent of the annual budget* 
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Source of Monies 



The board has supported the program almost entirely from district 
monies 55lnce Its Inception, In 1983-84, the district received some outside 
support In the form of a Ford Foundation grant totaling $445,000 over a 
five-year period. These monies have been used to provide supplemental 
support to the Schenley High Teacher Center for such activities as evalua- 
tion studies, and project documentation. The board has also set a goal to 
obtain approximately $l»000,000 from local, private, corporate and family 
foundations to support a three-year ancillary services program called the 
Support Services Project, This project has been proposed to strengthen 
several aspects of the entire secondary school Improvement effort, Including 
adjunct aspects of the Schenley Teacher Center program such as teacher 
externshlps In the private sector, and community relations. 



Staff Development; Content, Process and Evaluation 

Administrators: Content and Process 

PRISM 1 . Beginning In the 1981-82 school year, the staff development 
team Initiated the training of central office staff, building principals, 
selected vice-prlnclpals, and supervisors In PRISM I content. PRISM I was 
divided Into two stages. In Stage 1, all of the above staff received 30 
hours of Instruction In the following content: 

• teaching to an objective 

• selecting an objective at the correct level of difficulty 

• monitoring the learner and adjusting teaching 

• using, without abuse, the principles of learning 
— motivation 

— reinforcement 
— sequence 
— practice 

• five types of conferencing: 

— the A conference: observer reviews what went well and 

tells why 

— the B conference: observer Identifies what went wrong and 

gives alternatives 



*Conference types A, B, and C were stressed In Initial training. 
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— the C conference: teacher is asked to identify what went wrong 

and both teacher and observer generate 
alternatives 

--the D conference: observer identifies serious teaching 

errors and gives alternative corrections 

— the E conference: stretching the excellent teacher to do 

even better 

• anecdotal note-taking 

• analysis of teaching 

• lesson design 

• application of the model — teaching and conferring by participants. 

' Administrators and supervisors could select one of four training groups 
or cycles scheduled over the year at a centralized training center. Each 
training cycle involved about 40 participants and consisted of five one-day 
(six-hour) training experiences spread over the period of about a months 
Days one and two were offered consecutively. The remaining days were spaced 
over the toonth to afford practice of the model. Principals were directed to 
select two or three of their more effective teachers and practice observing, 
anecdotal note taking, lesson analysis « and conferring with these teachers 
both during and following training for the remainder of the year. Princi- 
pals were also directed to teach lessons to students during training, for 
practice, understanding, and application of the model. Each SDT team member 
was assigned roughly 25 principals and/or supervisors and regularly observed 
their practice observation/conferencing sessions as coaches and provided 
feedback. Central office staff, including the superintendent, received 
their training in 10 one-half day sessions spread out over the 1981-82 
school year. 

Training consisted of a variety of activities including PRISM theory 
and demonstration sessions, identifying effective and ineffective teaching 
behaviors, observing videotapes of lessons and taking anecdotal notes, 
practice in lesson analysis, and practice in teaching their peers and post- 
conferencing, as well as critiquing the conferences of others. In essence, 
year one training was designed specifically for the principals' growth in 
PRISM and the Hunter Model. Practice, with a small number of teachers and 
follow-up coaching by the SDT trainers were, key elements in the training 
experience. During PRISM I, Stage 1 training, principals were expected to 
assume the leadership for introducing PRISM in their own buildings and ex- 
plain to their staff the Stage 1 training they were undergoing. They were 
also expected to initiate the training of their own school staff in in- 
service sessions in the 1982-83 school year, with the assistance of 
materials and on-site help provided by the SDT trainers. 
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In the Slimmer of 1982, principals, selected vice-principals, and super- 
visors received ten days of additional clinical training by the SDT staff in 
Stage 2 of PRISM I, Stage 2, referred to as the P-T-O-C (Plan, Teach, 
Observe, Confer) cycle, involved principals in two five-consecutive-day 
workshops during which time they planned lessons and taught summer school 
students for one full period each day, observed/conferred the teaching of 
their peers, and were themselves observed/conferred on each daily lesson. 
Four blocks of five-*day workshops were offered and principals selected the 
workshop blocks which fit their schedule. Participants thus received 
additional practice in anecdotal note-taking, conferencing, observing and 
providing feedback, and practice teaching the content of the model. They 
viewed videotapes of themselves, role played and viewed training films. 
More importantly they experienced how the model could be applied on a daily 
basis and developed a sensitivity of the multiple decisions involved in 
providing daily instruction for students. 

The remainder of the district's deans, assistant principals and vice 
principals were trained in PRISM I, Stages 1 and 2 during the 1982-83 and 
1983-84 school years. Practice in teaching the model during Stage 2 
(P-T-O-C) of PRISM X training was done in regular classrooms during the 
year. 

All principals, selected vice-'principals, and supervisors were provided 
with additional PRISM I follow-up practice and training or coaching in the 
1982-83, 83-84, and 84-85 school years. In each of these years, in addition 
to assuming the leadership for providing inservice to their own school staff 
in PRISM I, they were expected to conduct a minimum of three teacher obser- 
vations and follow-up conferences per week, or 12 per month.* They were 
also visited, at least four times each year, by a SDT member who co-observed 
and analyzed a lesson, observed them post-conference the teacher, and pro- 
vided them with feedback on their conference to insure that they were appro- 
priately internalizing and applying the PRISM I instructional model and con- 
ferencing procedures. In the third year of the program, SDT staff spent 
more time with some principals; either those less skilled or new principals, 

PRISM II . Training of administrative staff (principals, vice- 
principals, deans, supervisors and central office personnel) in PRISM II 
commenced in the summer of 1983 and has continued each summer since then. 
The primary long term goal of PRISM II is to enhance the instructional 
leadership knowledge, attitudes and skills of administrators. Each summer 
since 1983, the program has consisted of a series of supplemental workshops 
on various topics offered over a ten-day period the first year and a five- 
day period in subsequent years. In the first year, the topics covered such 
areas as questioning skills, classroom management techniques, parent in- 
volvement techniques, PRISM I, Stage 1 review, additional conferencing 
styles (D&E), and school team planning by building administrators to prepare 
for PRISM I, Stage 1 inservice for teachers. 



*The 12 observations/conferences per month requirement was reduced to 
six per month in early 1985. 
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The program has evolved since the summer of 1983 to the point where, in 
the summer of 19S5 some 43 separate workshops or mini-coureee ranging in 
total time from one and one-half to three hours each were offered. Some 
workshops are recommended or required of various administrative staff, based 
on their role. Other workshops are optional. All administrators are re- . 
quired to attend five days of training each summer and to sign up for 
courses so as to completely fill each six hour workshop day (i.e., a minimum 
of two or a maximum of four courses can be taken each day over the cycle, 
depending on the length of the courses chosen). The courses offered this 
past summer were clustered in five major areas: staff assessment, super- 
vision, management, professional/personal growth, and curriculum. 

SDT staff are continually operationally defining what is meant by 
instructional leadership in the context of the definition presented earlier. 
The planning of the annual summer workshops has been a collaborative effort 
between SDT staff and representatives of various administrative groups to 
tailor the PRISM II program to both individually perceived needs and dis- 
trict needs. As was noted above, one of the district needs that has been 
addressed in PRISM II has been the development of building level action 
plans by administrative teams to orchestrate the delivery of PRISM I 
inservice to their staff. The action plans have typically considered: admini- 
strators improvement needs regarding PRISM I, procedures for collecting data 
on teachers' needs, areas where PRISM I needs to be reviewed, specific 
arrangements (who, when, where, how) to teach new PRISM I content to staff, 
and other areas of staff development related to ongoing district school im- 
provement efforts. In addition to providing the above structured experience 
and guidance to increase administrators' staff development skills, the SDT 
responded to a need to provide a structure for collegial support among admini- 
strators. Accordingly, several ongoing in-dlstrict regional networks were 
formed in the 1983-8A school year to provide principals with the opportunity 
to meet for one-half day, three or four times per year, to discx'ss and share 
PRISM implementation strategies and issues. 

In summary, district administrators received some 90 hours of intensive 
in-class training in PRISM I, and have since received 30 hours of additional 
training in PRISM II each summer since 1983, to supplement the leadership 
skills required to become both an instructional leader and an educational 
leader. They have also received intensive on-site follow-up coaching and 
monitoring regarding the new roles that they have been expected to imple- 
ment. In addition, beginning in the 1983-8A school year, secondary level 
administrators received four weeks of further o;rientation and training in 
connection with the start-up of the high school teacher center initiated 
that year. Similar orientation and training is planned for elementary level 
principals pending the start-up of the elementary school teacher center in 
the fall of 1985. 

Teachers: Content and Process 

PRISM I . The general training of the district's teachers K~12 in basic 
PRISM I content began in the 1981-82 school year. The first year was more 
of an introductory year, however, and principals worked with only a few 
teachers, primarily for their own benefit in acquiring the instructional and 
conferencing skills required by the model. The training of all teachers by 
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principals, vice-principals and supervisors, with the back-up support of SDT 
members, began in the 1982-83 school year. The inservice was accomplished 
via training activities, taught and coordinated by administrators, and 
through the principals' required twelve per month teacher observations and 
conferences* SDT staff provided support on request in the form of prepared 
instructional materials and on-site assistance with inservice. In 
preparation for their staff development role, each principal was initially 
required to assume responsibility for teaching four areas of the Hunter 
model. In later years they were held accountable for teaching all of the 
Hunter content* 

The training, observation and conferencing of school staff by building 
administrators K-12 continued through the 1983-84 and 84-85 school years to 
the present* A considerable portion of the district's five half-day 
inservice times for each school was allocated to PRISM training over the 
past three years. In addition to the allocated inservice days, all 
elementary teachers are allotted one period per day to meet with their 
building principal or instructional supervisor for the purpose of improving 
instruction. This period is called Essential Staff Educational Practice 
(ESEP) time. In many buildings, principals in cooperation with teachers 
have also scheduled PRISM training, review, and/or discussion on single or 
double ESEP periods on a regular basis, 

PRISM III . The Schenley High Teacher Center (SHTC) program, PRISM III, 
was initiated in the 1983-84 school year, k cross section of over 200 
school, business, and community personnel contributed to the design of the 
teacher center via various planning committees in 1982-83. The program was 
designed primarily tor (1) provide an in-depth clinical teaching experience 
for all the district's secondary school teachers, (2) provide teachers with 
an update in their subject area(s) and review the district's expectations 
for those content area(s), (3) provide them with a more in-depth under- 
standing of adolescent development and the relationship between students' 
development and academic behavior, and (4) afford them the opportunity to 
engage in independent study and/or internship activities to enhance their 
background for teaching. The program consists of three phases: (1) orien- 
tat ion at the home school, during which time teacher center staff, the 
teacher and the principal, plan a program to address the teacher's and the 
home school's needs; (2) an eight-week mini-sabbatical, during which the 
teacher participates in the model teacher center training activities; and 
(3) a follow-up phase, during which time a support structure is provided to 
assist the teacher in applying what has been learned at the teacher center » 

During the eight weeks at the teacher center, the visiting teacher: 
(1) observes exemplary teaching in an ongoing school setting; (2) practices 
new instructional techniques and skills; (3) receives detailed feedback and 
analysis on that practice; (4) applies instructional theory to practice; 
(5) receives an update in the content of his/her subject area(s) ; (6) re- 
views and discusses recent research on effective teaching, classrooms, and 
schools; (7) studies adolescent behavior and its implications for effective 
teaching; and (8) researches a chosen topic for personal growth. The over- 
all process involves participation in seminars with peers; the clinical 
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teaching experience; fulfillment of an individual study plan consisting of 
work with college, community and/or business personnel; and training in new 
technologies where appropriate, including computers. Follow-up activities 
consist of professional seminars by returning visiting teachers, 
participation in peer observations, PRISM content update sessions, and 
external-to-school activities (cross-school and/or business related visits). 

By way of structure or design, the Schenley High School is a fully 
functioning high school in all respects. The staff consists of a cadre of 
resident teachers (RTs) who teach four classes per day and engage in a 
variety of other regular school support activities as well as activities in 
support of the vioiting teachers; and a cadre of 24 clinical resident 
teachers (CRTs), about one-third of the total school staff, who teach three 
classes per day and also work in-depth with the visiting teachers. Each CRT 
works with two visiting teachers on intensive teaching clinics based on 
PRISM I*s principles of effective instruction over the eight-week period. A 
cadre of A8 professional replacement teachers are also a part of the pro- 
gram. They take over the visiting teachers' roles in the home schools while 
the visiting teachers are at the model high school for eight weeks. 

All staff for the program, RTe, CRTs and replacement teachers, were 
specifically recruited because of their effective teaching abilities. The 
RTs and CRTs received intensive training in PRISM I, Stages 1 and 2, by SDT 
staff prior to the start-up of the program. The replacement teachers were 
also trained in PRISM I, Stage L, SDT staff participate on a regular basis 
in the teaching clinics between CRTs and visiting teachers, in addition to 
teaching PRISM at the center. The overall management of the model high 
school teacher center is shared by the school principal who deals with 
regular school staff and curriculum matters, and the teacher center director 
who is responsible for the planning, design modification, and daily ope- 
ration of the program for the visiting teachers. 

The PRISM III high school teacher center experience was designed for 
all the secondary teachers in the district's 15 high schools. Four 
quarterly cycles of approximately 48 teachers each, have been scheduled 
through the program each year since September 1983, and some 200 teachers 
have completed the program to date. It is expected that all secondary 
teachers will have experienced the program by the end of the 1986-87 school 
year. PRISM III has also involved, beginning in the 1984-85 school year, 
two ten-day visitations to the teacher center by high school administrators 
to: (1) provide them with a first-hand, in-depth understanding of the 
training their staff are receiving; (2) afford them the opportunity to 
initiate a support network regarding PRISM III follow-up; and (3) give them 
a chance to review current theory and researci on supervision and educa- 
tional leadership, and explore leadership-related externships in the 
business community for their own benefit. Tlius, PRISM I, II, and III con- 
stitute a coordinated effort. 

The district's initial focus on the secondary level (PRISM III) was 
somewhat of a pragmatic decision on the superintendent's part. The superin- 
tendent perceived that the student dropout and declining enrollment problems 
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at the secondary level were more serious than any problems at the elementary 
level* The district was projecting the layoff of a large number of secon- 
dary staff in the early/mid-1980s due to the declining secondary enrollment ♦ 
Gearing up the Schenley High Teacher Center saved a number of staff posi- 
tions and thus served as an incentive for the Teachers' Federation to coope- 
rate with the improvement project. In essence > due to PRISM Ill's design, 
there is some double residency of teachers at the SHTC in addition to the 48 
teachers who were kept on board to serve as replacement teachers for 
secondary staff visiting the SHTC for eight weeks* Thus, some 60-70 secon- 
dary level teaching positions were saved. 

PRISM IV . The next phase of the improvement project, PRISM XV, will 
involve the opening of the Pittsburgh Elementary Model School (PEMS) in 
September 1985, to provide elementary school (K-S) staff with a clinical 
teaching experience similar to that currently being offered to secondary 
teachers at the high school teacher center, and further enhance elementary 
principals' instructional leadership skills. Planning for PEMS (PRISM IV) 
began in the spring of 198A with the formation of an administrative steering 
committee which included SDT members* A planning task force comprised of 
three major committees (instructional process, school organization » and 
formative evaluation) and numerous sub-committees was charged with planning 
PEMS* Teachers and other support staff were brought into the planning 
process in September 1984. Planning and preparations continued through the 
1984-85 school year. The district expects that all elementary school staff 
will participate in PEMS in six training cycles per year over the next 
three-and-one-half years* 

Similar to the model high school teacher center, PEMS will involve RTs, 
CRTs, and replacement teachers* However > some organizational differences in 
design are planned. Elementary teachers will spend seven days with the SDT 
at the district's central training center » followed by ten days at the model 
elementary school. Three days of formal teacher follow-up by the CRTs and 
SDT staff in the teachers' home schools are also planned along with struc^ 
tured peer observation at the home school^ for a total of 20 days (4 weeks) 
of systematic PEMS» PRISM IV training experience. 

The program was shortened because it was felt that elementary teachers 
could afford to spend less time away from their more dependent students than 
secondary teachers; and the elementary teachers already had a stronger PRISM 
I foundation than the secondary teachers due to the fact that elementary 
principals have traditionally been more active in instructional leadership, 
and due to their increased exposure to PRISM I via the use elementary staff 
have made of their scheduled ESEP time over the past few years. 

It also bears noting that the district decided to save the middle- 
schools for last, regarding more in-depth PRISM training. The middle 
schools are newer and their staff have had more recent training and more 
training than either elementary or secondary level staff* 
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Evaluation of Staff Development 

From the very inception of the PRISM project, the district incorporated 
systematic evaluation procedures to monitor, adjust and evaluate the effec- 
tiveness of all aspects of PRISM* The evaluation work has been carried out 
by the district's Office of Testing and Evaluation (OT&E) and other Pittsburgh 
area college and university staff, with major support also being provided by 
the Evaluation Unit of the Learning Research and Development Center (LRDC) 
of the University of Pittsburgh. PRISM evaluation has focused on three 
major activities: (1) the maintenance by LRDC of archival descriptive 
records, field notes and documentation of program planning, development and 
implementation activities, to provide a chronological record of program 
decision making and implementation to inform program leadership staff and 
facilitate subsequent planning; (2) the conduct of formative evaluation 
activities by OT&E and LRDC staff (e.g., surveys, interviews, critical an-- 
alyses of materials and on-site observations of program implementation) to 
provide data to program staff to assist them in continuously modifying and 
improving the program; and (3) the conduct of impact studies by OT&E staff 
and others (e.g., surveys, tests, quasi-experimental research, value assess- 
ments, attitude changes, classroom observations) to determine if the program 
is having the intended effects on teachers* and administrators' knowledge, 
attitudes and behaviors. 

The purpose of this paper Is not to describe in detail the evaluation 
of PRISM, but to provide an overview or sampling of the types of evaluation 
activities conducted. For example, all PRISM training activities have 
included an end-of-training survey of participants' perceptions of the 
effectiveness and utility of training. Following IltlSM I training, 
principals were asked to rate the degree to which the workshop was well 
organized, met the objectives, helped to increase their understanding of the 
model, and would help them to function more effectively. They were also 
asked to comment on the most valuable things they learned, specifically how 
the model would help them improve their teaching or conferencing, how the 
session leaders could be more effective, whether they desired more training 
or support and in what areas, and their general suggestions for additional 
workshops and for improving the workshops. 

Post-training surveys have also been conducted following the training 
sessions for teachers. Similarly, exit interviews have been conducted with 
the visiting teachers at the high school teacher center to tap their per- 
ceptions of the program's utility and their recommendations for changes in 
the program. In addition, data have been collected on tho implementation of 
the required teacher observations/conferences and the required PRISM in- 
service of teachers in their home schools. These formative evaluation 
activities have assisted SDT and other core staff in making a number of struc- 
tural changes in the PRISM programs to improve their effectiveness and 
utility. 

A number of other short-tetm and long-term studies either have been 
conducted, are in progress, are planned, or are ongoing efforts. These 
studies will or have focused on specifics related to: (1) the efficacy of 
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the replacement and visiting teachers, replacement process; (2) behavioral 
ctianges in visiting teachers' classroom performance with respect to PRISM 
behaviors and time-on-task followir.g attendance at the teacher center; 
(3) the efficacy of the RT, CRT, visiting teacher process and problems 
attendant to the new roles and relationships; (4) visiting teachers changes 
in attitudes toward students, the profession, etc; (5) model high school 
teacher center students' perceptions of the school climate; (6) changes in 
the academic performance of the students at the model high school; 
(7) visiting social studies teachers* progress in implementing the compon- 
ents of a critical thinking program; (8) principals' perceptions of their 
involvement in the high school teacher center program, of related follow-up 
activities, and the selection-program process for visiting teachers; (9) the 
effects on home school students of the visiting teacher/replacement teacher 
process; (10) principals' perceptions, over time, of the efficacy/utility of 
PRISM training and its impact on schools, in part. In the context of the 
original district needs assessment; and (11) the implementation and per- 
ceived utility/ fairness of the new end-of-the-year , formal teacher evalua- 
tion procedure, and the effects of PRISM on said procedure. 

The studies that are ongoing or that have been completed have generally 
found either positive outcomes or outcomes that were developing in the in- 
tended direction* They have also identified implementation problems for SDT 
and SHTC staff consideration and action. For example, a study of princi- 
pals' perceptions of PRISM' s effects reported by LRDC staff in September 
1984, found that district professionals: (1) had a clearer understanding, 
systemwide, of instructional expectations; (2) found the criteria for evalua- 
ting teacher instructional effectiveness to be clearer; (3) perceived a Utore 
effective articulation of instruction across levels; (4) found the inservice 
to be more relevant to building needs; and (5) were ready to take on new 
skills related to curriculum needs* The report also cited perceived pro- 
blems, cited here in part, with PRISM paperwork, the he^ivy observation/ con- 
ferencing schedule, finding time for curriculum work, and a desire for even 
more tailored inservice sessions for administrative staff. 

Evaluation therefore has been and continues to be an ongoing component 
of PRISM planning, implementation, and follow-up* In the words of the 
superintendent (Wallace, 1985), "the general results (of the evaluation) 
..ndicate an unpredictably high level of enthusiasm for the program. The 
data also indicate that principals are taking the program seriously. Data 
gathered with respect to the implementation of PRISM II indicate that about 
one-third of the principals in the district have embraced and to some extent 
put into operation the concepts implicit in the model. Approximately 
another third of the principals are still ,-:truggling with many aspects of 
the model. And about one-third of the administrators are trying very hard 
to avoid the concept hoping that the expectations will somehow 'go away'," 
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The following chart sutranarizes the overall staff development schedule 
and timeline for PRISM* 



Staff Development Schedule/Timeline 
rTmplementation Phases) 



Month - Year 

September 1981 
June 1982 



PRISM Training Provided; 
Content » Audience and Process 

• PRISM I, Stage 1 instructional 
leadership training for all 
principals and supervisors 
(mandatory) 

— five, five-day training 
cycles each spread over 
one month* Training in 
instructional theory and 
clinical supervision^ 
Coaching of each trained 
administrator on site as 
he/she worked with 
selected teachers* 



Approximate 
Number Trained 

200 Principals 
and Supervisors 
(30 hrs) 



600 Teachers* 
(5 hours of 
PRISM I. Stage 1) 



July 1982 



September 1982 
June 1983 



PRISM I, Stage 2 P-T-O-C 
(Plan, Teach, Observe, 
Confer) training of 
principals during summer 
school . 

Mandatory weekly observations 
by all principals and super- 
visors (accompanied by 
coaching of administrators on 
site) along with initial 
training of school staffs by 
administrators* . , (continued 
each year to the present 
date) * 



84 Principals 
(30 hrs)~ 
(ten days, three 
hours each day) 



3,000 Teachers 
(4 hrs of PRISM I, 
Stage 1) 



Mandatory PRISM I, Stages 1 & 
2 training of vice principals, 
deans » and new administrators, 
plus PRISM I, Stage 2 P-T-O-C 
training of supervisors. 



40 Vice-princi- 
pals and Deans plus 
12 New Adminis- 
trators (60 hrs) 
65 Supervisors 
(30 hrs) 



*The time figure(s) cited for training teachers in PRISM I, Stage 1 
represent the mode ac-oss schools. 
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Staff Development Schedule /Timeline 



(Implementation Phaacs) 



Month - Year 



PRISM Training Provided: 
Contents Audience and Process 



Approximate 
Number Trained 



July 1983 



Initiation of mandatory 
PRISM II instructional 
leadership supplemental 
training of administrative 
staff primarily in leadership 
knowledge, attitudes, and 
skills other than PRISM* 
Some required and some 
optional courses: 
(e.g., questioning skills, 
communication skills, school 
team planning for inservice, 
additional conferencing 
styles. Stage 1 review, 
curriculum content, principal 
networks) , , .Continued each 
summer to the present date.,, 
43 mini-courses or workshops 
offered in the summer of 1985. 



200 Administrators 
(30 hrs each 
summer since 1983) 



September 1983 
June 1984 



Continuation of weekly 
observations by all 
administrators plus on-site 
coaching. 

Continued training of school 
staffs. 



PRISM I, Stage 1 training for 
district counselors. 



3,000 Teach erF 
(4 hrs of PRISM I, 
Stage 1 review) 

85 Counselors 
(30 hrs) 



PRISM 1, Stages 1 & 2 
training of new admin- 
istrators. 



15 Administrators 
(60 hrs) 



PRISM I, Stage 2 P-T-O-C 
training for assistant school- 
based administrators. 



40 Administrators 
(30 hrs) 
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Staff Developinent Schedule /Timeline 



(Implementation Phases) 



Month - Year 



PRISM Training Provided: 
Content , Audience and Process 



Approximate 
Number Trained 



September 1983 
June 1984 
(continued) 



September 1984 
June 1985 



PRISM III High School Teacher 
Center program initiated. 
Four, eight-week training 
cycles per year for visiting 
secondary school teachers. 
Training in PRISM I, teaching 
clinics, peer conferencing 
models, adolescent behavior, 
content area update, indivi- 
dual study. Includes two, 
ten-day orientation and study 
experiences for secondary 
princ ipals. 

PRISM I, Stages 1 & 2 training 
of new administrators. 

Continued training of school 
staff. 



192 Secondary 
school teachers 
(30 hrs of 
PRISM I, Stage 1 
plus considerable 
additional train- 
ing) 



15 Administrators 
(60 hrs) 

3,000 Teachers 
(4 hrs of PRISM I, 
Stage 1 review) 



PRISM III High School Teacher 
Center program continued. 



192 Secondary 
School Teachers 
(see above) 



-In addition to ongoing 
informal evaluation, 
initiate formal evaluation 
of teacher center. 



40 Intermediate 
Unit Teachers 
(see above) 



September 1985 
June 1986 



PRISM III High School Teacher 
Center program continued. 



• PRISM IV Elementary School 
Teacher Center program to be 
initiated. 
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Implementation of District Supervision/Evaluation System 
Philosophy /Pol Icy 

The district's philosophy regarding teacher supervision and evaluation 
are presented best In a paper by the superintendent (Wallace, 1985), 
juxtaposed sections of which are cited below: 

Personnel evaluation was established as the district's 
second highest educational priority • In so doing, the Board 
of Education reflected Its own views as well as those of 
cotmnunlty members and school district employees. Essent- 
ially, the survey data revealed that respondents believed 
that too many teachers and administrators were not per- 
forming their duties effectively and this condition needed 
to be corrected. (p»l3) 

The superintendent perceived that two alternatives were 
available In responding to this priority* The first alter- 
native would have been to use the existing evaluation 
systems and embark on a "wltcb hunt" to Identify the in- 
effective personnel and then seek to demote or discharge 
them. The second alternative would be to seek to increase 
the quality of supervision and evaluation and set out to 
improve the performance of all personnel in the district; 
this approach would require that the performance expec- 
tations for all personnel be carefully detailed and that 
persons be observed and provided with structured feedback to 
Improve performance. The first alternative is clearly puni- 
tive in nature and would likely produce negative side 
effects among teachers and administrators* It would pro- 
bably create an atmosphere of negativism that could prove 
detrimental to the more positive improvement thrust of the 
Board. The second alternative is Improvement oriented and 
Is designed to make good teachers and administrators better 
while at the same time identifying those who need signifi- 
cant Improvement. The latter approach would still induce 
some anxiety among teachers and administrators, yet it would 
be approached in a constructive spirit of providing persons 
an opportunity to Improve their performance* The latter 
approach places professionals in a helping relationship with 
one another to bring about a positive improvement in the 
state of educational affairs. The constructive approach was 
selected to Improve personnel evaluation procedures and the 
general level of professional performance in the district. 
This plan became known as PRISM-Pittsburgh Research-based 
Instructional Supervisory Model, (pp. 13-14) 
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PRISM I has four essential components: (1) knowledge train- 
ing; (2) skill development; (3) follow up coaching; and (4) 
colleague networks. PRISM I is based on the following as- 
sumptions: (1) that personnel evaluation will be enhanced 
when teachers, administrators, and their evaluators are en- 
gaged in a dialog that focuses on clear communication of 
expectations for job performance in that role; (2) that a 
consistent framework of effective teaching based on research 
findings exists and can be taught, learned and applied; (3) 
that teachers, administrators, and supervisors can be 
trained to observe performance, gather evidence with respect 
to that performance and provide structured feedback that 
will cause that performance to be improved; (4) that if 
teachers and administrators are unable to improve their per- 
formance after careful role clarification, reasonable obser- 
vation and feedback, and specific training, then action will 
be taken to terminate their employment, (p. 14) 

PRISM II is the district's program to improve the instruc- 
tional leaf^^ership skills of principals, supervisors and 
central office personnel. PRISM II has been developed be- 
cause most principals have not been trained as instructional 
leaders; degree and certificate programs for administrators 
have tended to focus primarily on the management aspects of 
schooling. Many administrators are not prtpared to cope 
with the current emphasis on instructional leadership. Not 
only has their training not prepared them to assume this 
role, most school boards and school districts have not ex- 
pected this. Principals, often are selected for their 
positions because they are good at public relations or good 
at discipline. More often than not, supervisors of instruc-. 
tion at the elementary, middle and secondary level are 
better prepared to offer "content centered" instructional 
leadership. However, they often lack the status and the 
power to exercise potent leadership. Thus with the new em- 
phasis on educational improvement, the nation finds its 
schools staffed with many principals who are not well pre- 
pared to assume this new instructional leadership role, 
(pp. 17-18) 

PRISM II is based on the following assumptions: (1) in- 
structional leadership can be defined, implemented, and 
evaluated; (2) all principals can become instructional 
leaders; (3) most principals will need substantial training 
in order to develop the knowledge-base and the skills to 
provide instructional leadership; (4) the process of deve- 
loping instructional leadership can be facilitated by estab- 
lishing collegial networks of administrators. The training 
workshops, and the coaching of PRISM I serves as the foun- 
dation for PRISM !!♦ Above and beyond PRISM I, however. 
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principals and other administrators must have a knowledge- 
base with regard to curriculum models and instructional 
techniques* Principals need to know enough about organi- 
zational development and the educational change process to 
furnish an environment for teachers that is likely to pro- 
duce a focus on instruction, (p. 18) 

The High School Teacher Center (PRISM III) is the Pittsburgh 
School District's response to the Board of Education's pri- 
ority to increase the effectiveness of instruction at the 
secondary level. It also reflects the district's need to 
reduce the high school drop out rate* The Teacher Center 
program is based on the following assumptions: (1) that 
secondary teachers can be engaged productively in a 'clinical 
experience* that will cause them to reflect upon and improve 
their teaching techniques as they observe other teachers, 
analyze instruction, teach and receive feedback on their 
own instructional techniques; (2) that we can develop an 
instructional dialog that will tend to break down the instruc- 
tional isolation experienced by most secondary teachers; (3) 
that by providing opportunities for teachers to participate 
in lectures and seminars they will upgrade their skills and 
knowledge in their content area; (4) that by participation 
in seminars on the adolescent, teachers will gain greater 
understanding of and increased skill in dealing with today's 
urban youth* (pp* 20-21) 

Implicit in the district's philosophy and staff development approach 
to teaci er supervision and evaluation is the superintendent's philosophy of 
educational leadership. In the paper cited above, Wallace commented as 
follows on his views of educational leadership: 

Leadership can be defined in many ways* Expressed very 
simply, it can be described as a process of working with and 
through other people to get a job done* Educational leader- 
ship, at the level of the superintendent, requires extensive 
goal setting, planning, implementation and evaluation re- 
lated to instruction. The author views the following as 
some of the key components of effective educational leader- 
ship by the superintendent of education: 

• Educational leadership must be data driven* 

• Participative planning is critical. 

• Respect must be communic.ited to teachers and iriacipals 
who develop programs. 

• Risk taking is essential if progress is to be made. 
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Knowledge of the change process is important. 



• One must have a vision of good education, 

• Follow through is essential, 

• Recognition of the key role of principals in school 
Improvement is vital, 

• Routine administrative matters must be delegated, 

• Xhe superintendent must model the instructional 
leadership behavior that will be expected of principals 
and other administrators. If principals observe the 
superintendent engaged in data analysis, planning, 
developing. Implementing and evaluating instructional 
initiatives* they can adapt those behaviors to their own 
responsibility. If the superintendent models the 
behaviors listed above, the stage is set for 
administrators to play a similar role, (pp, 29-31) 

The district's philosophy of teacher supervision and evaluation, and 
the central role that staff development plays in improving that process in 
the context of PRISM are further reflected in the following partial excerpts 
of a paper. Two years Into PRISM (Marshall and Radvak, 1983), by SDT staff: 

The monitoring component of the program in the schools has 
come to be called "coaching on site.*' It is this role that 
the SDT feels is the key ingredient in the ultimate accep- 
tance of PRISM in the schools. It is essential to the 
process that one grasps the significance of the concept of 
coaching — that successful use of newly acquired techniques 
require practice, and that one may be working on only one or 
a few development areas at a time. It is necessary to break 
down the historical view of in-service which says that once 
something happens in training, the expected change in be- 
havior transfers automatically and completely to the work 
site. The SDT tries to provide feedback in appropriate in- 
crements to the administrators; and administrators, in turn, 
try to remember to deal with circumscribed areas each time 
they confer with teachers. 

A problem that the SDT has encountered is to convince 
others that, because 250 administrators "have gone through 
the training" and 3,000 teachers "will have been taught 
PRISM," it is not thereby time to move on to something else. 
The complex task of analyzing and discussing teaching for 
the purpose of positively reinforcing and practicing the 
multitude of sound teaching behaviors or of modifying less 
desirable ones is a very challenging process and requires 
time. 
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In this freeze-dried generation where change over time 
is an alien concept, it is difficult to slow down the 
process so that in fact the transfer of newly acquired or 
refurbished skills from the knowledge to the application 
level will occur with administrators and teachers alike • 

In conjunction with the time element, there is the 
level of trust that needs to be developed between the SDT 
and the administrators in training and between the adminis- 
trators and their teachers. Moving from a traditional 
summative procedure (in which teachers are observed once a 
year for purposes of evaluation) to a formative one (in which 
frequent observations and conferences take place for purposes 
of development) requires the belief that the intention is to 
improve instruction not to get rid of instructors* 

That belief is indeed occurring thanks in large measure 
to the skills and commitment of the Pittsburgh administrators. 
They have taken the positive message of PRISM to their staff 
members who are joining with them in, as one teacher put it, 
thoughtful "informal professional dialogue" about effective 
instruction ♦ 

One night argue that staff development is simply an 
on-going function of supervision and that there's nothing 
especially urgent about developing new models; but the 
limited growth and resources familiar to our times would 
seem to indicate that it must be at the forefront of educa- 
tional thought and activity ♦ The era of staff development 
is here* Yew school staffs have a majority of new teachers. 
The older among us have not been in college classrooms as 
students for a long time. New ideas about learning and the 
learner have emerged. A staff development program is one 
way to keep a school system not only current, but reaching 
into the future, (pp, 4-5) 

Requirements 

Supervision * Starting in September 1982-83 through February of the 
1984-85 school year, the district required that all principals conduct 12 
PRISM-related classroom observations /conferences per month. Through the 
greater part of the first year principals focused on A, B, and C-type 
conferences; D and E-type conferences and the refinement conferences were 
introduced toward the end of the 1982-83 school year. In the refinement 
conference the principal and teacher discuss "what went well and why." They 
also discuss "factors which may have impeded the progress of the lesson" and 
determine specific procedures to "add a piece to the teacher's teaching," to 
address those factors which impeded progress. Follow-up is also planned. 



77 

77 



ERIC 



BEST COPY AVAILABLE 




Generally, the announced or unannounced PRISM observations /conferences 
conducted in the first year-and-a-half were brief; I0-L5 minutes of class- 
room observation followed by LO-15 minutes of conferencing. The emphasis 
was also primarily on the positive; what the teachers were doing well. It 
was the district's philosophy that, given the large number of required ob- 
servations, principals would get to see each teacher a sufficient number of 
times over the year, to get a picture of his/her performance* The district 
also wanted teachers to receive positive feedback to build trust. The 
number of times teachers have been observed each year has ranged from 3 or 4 
to 10 or 12 depending on the size of the school's faculty. 

In the 1984-85 school year, principals were directed to conduct more 
refinement conferences in order to begin to address more systematically and 
more in-depth the needs perceived in teachers' performance. The nimiber of 
required conferences was reduced to six per month, in February 1985, in 
order to permit principals to engage in these refinement observations /con- 
ferences which each take from one to one-and-one-half hours. Principals 
were also directed to identify one or two teachers as "focus teachers" and 
to work in-depth with these staff over the year to enhance their teaching 
skills. It should be noted that no formal records of the content of indi- 
vidual teacher observations /conferences have been shared with central office 
staff to date. The PRISM Observation and Conference Logs^ and anecdoted 
observation notes are kept by principals in their own files and are not sent 
'*up-town," nor are they a part of a teacher's permanent file. Similarly, 
"focus teachers" are not publicly identified and are worked with unobtru- 
sively by principals. The focus of PRISM has been on training and helping 
and not on accountability for annual evaluation purposes. 

Evaluation , For the purposes of annual accountability the district has 
used the State required DEBE-333 evaluation form over the past several 
years. The district requires that non-tenured employees be observed/ rated 
twice a year and tenured employees once a year. Approximately 90% of the 
teachers are tenured and are rated once a year. Observations conducted for 
the purpose of annual rating are announced as such. District staff noted 
that it is difficult to separate teacher supervision from teacher eval- 
uation. The data from the supervisory observations do feed into the end- 
of-the-year teacher evaluations. The purposes of the supervisory obser- 
vations, however, are broader than evaluation. They are improvement 
oriented and involve follow-up. They noted that teachers are also expected 
to teach certain curriculum and that enters into their DEBE-333 evaluation. 

The Observation and Conference Report for Professional Employees and/or 
the Critical Incident Report forms used by the district for annual evalua- 
tion, in support of the DEBE-333, differ in format from the PRISM Obser- 
vation and Conference Log, In cases where ^'below average" or "unsatis- 
factory" ratings are given, at least three formal observations of reasonable 
length (e,g,, at least one-half period each) are required along with the 
supporting observation reports* PRISM Observation and Conference Logs are 
not to be used. However, support offered to teachers through PRISM can be 
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cited^ When teachers are rated as satisfactory, no supporting observation 
report data are required to be sent to the central office. Anecdotal cita- 
tions for exemplary teacher performance are encouraged but not required by 
the district ♦ Two unsatisf actories in a row constitute grounds for dis- 
missal of a teacher. All teachers receive copies of their DEBE-333 ratings 
and Observation and Conference Reports, and may arrange to discuss them with 
the principal. Ratings are not grievable. Teachers may, however, file a 
rebuttal which becomes part of the teacher's permanent personnel file. 
District staff also noted that teacher evaluations have played no part in 
any district layoffs. 

Over the period of the 1984-85 school year a district committee in- 
cluding teachers, federation representatives, central office personnel, and 
SDT staff worked to revise the standard DEBE-333 state form and obtain the 
state's approval of the modifications. The district plans to commence using 
the new annual evaluation form, which more accurately reflects the dis- 
trict's instructional emphasis via PRISM, in the 1985-86 school year. The 
merger of PRISM instructional concepts/terminology with the annual evalua- 
tion instrument represents a significant milestone in the district's imple- 
mentation of the TS/E improvement effort. It bears noting, however, that 
the district waited some four years, allowing adequate time for program 
growth, acceptance and maturation, before formally effecting the planned 
merger. 

Organization and Process 

The district's in-school supervision process is carried out by the 
school principals, assistant or vice-principals, supervisors and division 
heads* All these staff are active in carrying out various aspects of PRISM 
observations/conferences. The observation load in a given school ranges 
from 19 to 25 teachers in elementary schools to 100 to 200 staff In some 
secondary schools* As was previously noted supervision related staff deve- 
lopment activities have primarily been the responsibility of the SDT staff 
with the assistance of the CRTs in the model high school teacher center. 
Principals and their assistants have had the additional responsibility for 
PRISM staff development in their schools • The above school-based staff are 
expected to spend approximately 20% of their time on supervisory relate . 
activities. 

Formal, regular classroom teacher evaluation for accountability pur- 
poses is the primary responsibility of the principals. Principals, however, 
luay use the evaluation reports of other supervisory support personnel (vice 
principals, supervisors, etc.) to supplement their own annual teacher 
ratings, especially in cases where below average or unsatisfactory ratings 
are involved. The district director of personnel and employee relations has 
the formal management responsibility for the district's overall personnel 
evaluation system. 

As noted earlier, SDT members engage in three primary activities; 
follow-up coaching of administrators, teachtng PRISM content , and special 
activities related to their respective coordinator roles ♦ They estimate 
that they spend approximately one-third of their time in each of the three 
areas • A chart outlining their organization and division of labor is pre- 
sented below^ 
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PRISM Staff Development Team: Organization, Roles/Actlvltles 



Follow-up 
Team Coaching of 

Members Administrators 

A Coaches 14 high 

school principals 
and 30 high school 
vice principals 
times per year) 

Also monitors/ 
assists with 
beglnlng-of-year 
plans 



Teaching/ 
PRISM Clinics 

Teaches PRISM 
and conducts 
teacher clinics 
at the Schenley 
High Teacher 
Center (SHTC) 



Special Roles/ 
Activities 

Coordinates 
System-Wide 
Implementation, and 
Follow-Up for the 
SFTTC 

Involved heavily in 
initial Internal 
PRISM planning and 
curriculum develop- 
ment 



Working with the in- 
structional chair- 
persons at the 
high school. ' Will 
train them this 
sunnner to assume 
PRISM improvement 
roles 

Conducts seminars 
on "transfer" and 
lEPs 



Coaches all middle 
school and special 
school principals 
« 19 buildings 
(A times per year) 

Also monitors/ 
assists with 
beginnlng-of-year 
plans 



- Teaches PRISM and 
conducts teacher 
clinics at the 
SHTC 



Teaches new groups 
of administrators 

Coordinates PRISM 
evaluation and docu- 
mentat ion • Re spcn» 
slble for liaison 
with internal/ 
external evaluation 
staff and for 
observation and 
Inservlce reports 

Active on committee 
revising the DEBE-333 
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Follow-up 
Coaching of 
Administrators 



Teaching/ 
PRISM Clinics 



Special Roles/ 
Activities 



Coaches 18 ele- 
mentary school 
principals and 
vice principals 
(4 times per year) 

Also monitors/assists 
with beginning-of- 
year plans 



- Teaches PRISM and 
conducts teacher 
clinics nt the 
SHTC 



Developed/ teaches 
units on "questioning 
skills and classroom 
management'* 

Conducts seminar on 
concept attainment 

Documents SHTC 
interactions 
and suggests 
improvements 

Worked on refining 
the "teaching clinic" 
at SHTC (i.e. , group 
cbservat ion/analy^ is/ 
feedback of a teaching 
episode) 

Coordinates the 
development/ 
conduct of PRISM II 
Inservice Instruc- 
tional Leadership 
Workshops for 
principals 

Training 4 admini- 
strative assis- 
tants to help vice 
principals when 
school principals 
visit the SHTC for 
20 days 

Working on a resource 
book to give admin-- 
istrators alter- 
native ways of 
addressing fixed 
management tasks in 
the district 
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Team 
Members 



Follow-up 
Coaching of Teaching/ 
Administrators PRISM Clinics 



Special Roles/ 
Activities 



D - Coaches 18 ele- 

mentary school 
principals and 
vice principals 
(4 times per year) 

- Also monitors/ 
assists with 
beginning-of-year 
plans 



- Teaching role at 
SHTC 



- Coordinating the 
planning/develop- 
ment of the 
principal ' s 
experience at the 
proposed Pittsburgh 
Elementary Model 
School (PEMS) (i.e., 
administrator growth 
and development) 

- Conducts 2 seminars 
at the SHTC 



- Developing a trans- 
cript/management 
system of everything 
that has been offered 
regarding PRISM and 
of "who's taken what" 



- Helped develop and 
teach a workshop 
on class inanagement 
and discipline at 
the SHTC 



- Provided 30 hours 
of PRISM instruction 
to principals not 
hit earlier 



- Serves as liaison 
for business/edu- 
cation parttier- 
ships^-extertiships 
for SHTC partici- 
pants 
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Follow-up 

Team Coaching of Teaching/ Special Roles/ 

Members Administrators PRISM Clinics Activities 



E* - Coaches 12 ele- 

mentary school 
principals (4 
times per year) 

Also monitors/ 
assists with 
beg innlng-of -year 
plans 



Teaches PRISM I 
content 



Serves as a resource 
person for the four 
elementary prin- 
cipals' support 
network clusters 

Working on the 
development of an 
instructional pack- 
age on "grouping 
procedures" for 
secondary school 
teachers and super- 
vlsors* Will pilot 
in Sept* 1985 



Developed a PRISM 
Primmer book as a 
resource for 
principals and 
supervisors 

Worked on the 
design of the PEMS. 
Will help train 
PEMS staff 



SDT 

Project 
Director 



Helped with the 
design of the 
questioning and 
discussion skills 
inservice packet 

Responsible for 
coordination of the 
SDT: management, 
budget, liaison and 
communications • 
Sits on the 
superintendent * s 
cabinet 



*SDT team member E has been with the SDT since January 1985, when an 
original SDT member took a sabbatical* Members A, B, D, and the director 
have been with the teain since its inception* SDT member C was an 
additional team member added i October 198^* 

4 
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System Monitoring 

The district has employed an extensive monitoring system, to track the 
Implementation of PRISM. Principals have been required to submit monthly 
observation reports to the SDT, The monthly reports summarize the total 
number and type of observations/ conferences conducted, the number of 
teachers observed and the number of times each was observed, the subject 
areas In which the observations were conducted, and the PRISM content areas 
most frequently discussed In the conferences. They are also required to 
submit mid-year and end-of-the-year progress reports on PRISM, and other 
staff development activities carried out In their schools. These latter 
reports describe: (1) the school's progress In achieving the goals outlined 
in the schools' annual PRISM plan; (2) the district support staff used by 
the school and their specific role(s) In the Inservlce program; (3) the 
principals' PRISM networklng/colleglal support activities and comments/ 
reactlons/recOTamendatlons thereto; (4) the PRISM II Instructional leadership 
activities In which tha principals participated; (5) the timeline, personnel 
Involved, ^nd follow-up of PRISM inservlce training in ten specific PRISM I, 
Stage 1 areas; and (6) the principals' general assessment of the impact of 
PRISM in the school along with recommendations — accompanied by sample 
agendas, handouts, objectives and evaluations. Principals also describe 
their PRISM activities in their annual MBO report to the deputy and 
associate deputy superintendent. 

Members of the SDT have also maintained Staff Development Coaching 
Logs to document their coaching activities with principals. SDT staff log 
tha particular types of coaching activities they engage in, in each coach- 
ing session or visit — that is, checking the principal's observation/con- 
ferencing log, notes and dates; co-planning the conference; assisting with 
anecdotal note analysis; observing the conference; coaching the confer- 
encer; assisting with inservlce planning; reviewing the school's PRISM 
plan, and/or teaching fjpecific content to select school staff. The 
specific objectives of the coaching sessions, significant coaching session 
activities, and the planned follow-up activities of all parties Involved 
are also documented. When assisting principals and other staff with the 
conduct of refinement conferences, SDT staff provide feedback on the 
quality of the conf erencer 's preparatior./data analysis, introduction/ 
climate formation, diagnosis/perception check, reinforcement/skill main- 
tenance focus, and teaching/skill development focus activities via a struc- 
tured feedback form. 

The coaching data ulted above are confidential, as are the principals' 
PRISM observations of teachers, and are used to monitor/adjust coaching 
activities and assess needs for additional formal training in PRISM I and 
II. In 1984--85, a more formal attempt by SDT staff to rate principals' 
overall Instructional leadership behaviors, using a structured Instrument, 
was met with considerable resistance by the principals — even though the 
Intent was formative in nature* An additional task of the SDT, over that 
year, was to compile a record and set up a system to describe all the train- 
ing related to PRISM I, II and III that every staff member in Che district 
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has experienced to date. Finally, SDT staff have also used a 50-iteni PRISM 
Knowledge Asses-sraent quiz to assess staffs' long-term retention of PRISM I» 
Stage 1 concepts (25 items are true-false, the remainder are constructed 
response). Principals took the quiz in the summer of 1984 and teachers 
received It in the fall of 1984. The quiz results gave SDT staff an over- 
view of staffs' cognitive acquisition of PRISM concepts and identified 
areas where additional review was needed. 



Administrators' Perceptions 

Two high school principals, including the principal of the high school 
teacher center, three middle (6-8) and five elementary school principals 
were interviewed for their perceptions of the district's teacher 
supervision/evaluation system. 

Relationship Between Teacher Supervision and Teacher Evaluation 

The principals generally made a clear distinction between the helping 
role of the PRISM supervision process and the accountability role of eval- 
uation. Basically » they saw the two processes as being related in content, 
but separated in time, purpose, and process. Although they indicated that 
the data collected in PRISM observations for supervision purposes in- 
fluenced their teacher evaluation judgments, they still perceived super- 
vision as improvement oriented. 

Some typical comments included these. "PRISM is not an evaluative 
tool. It's a supervisory model to promote learning and instruction. One 
(supervision) leads to the other (evaluation) though." "The goal (of 
supervision) is to help teachers do a better job; to provide help before 
due process* The DEBE-333 and the observations are related." "The purpose 
of PRISM is to improve instruction. The DEBE-333 won't improve • instruction. 
No records (for accountability) are kept of PRISM observations." "We're 
instructed to supervise for the explicit purpose of improving teachers; all 
can improve. I see them as related though; evaluation follows supervision." 
"They somehow blend together but the PRISM model has made all staff con- 
scious of when we're observing for supervision and when we're evaluating. 
Our former process wasn't related to instructional practices and missed 
(omitted) the positive reinforcement of teachers. PRISM lets teachers know 
what they are doing right. Now they are less anxious about evaluation." 

Utility of PRISM I and II Training 

Without exception the principals praised the utility and quality of 
the PRISM training they had received. They commented positively on such 
things as: the development of a common language of instruction; the guide- 
lines for helping to improve instruction; the concrete modeling and prac- 
tice; the coaching involved in training and in follow-up; and the ongoing, 
in-depth, high quality of the SDT refresher workshops and summer workshops • 
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Over a third commented on their increased confidence in supervising 
teachers to improve instruction, and half expressed a desire to participate 
in refresher workshops and/or any new workshops the SDT might develop on 
new R&D in teaching. However, two or three indicated that the district was 
approaching the "overkill point" regarding PRISM training. 

Feasibility of Implementation 

All of the principals rr^ported that they generally were comfortable in 
their dual role as teacher supervisors and teacher evaluators. All also 
described their implementation of the process pretty much in accord with 
the district's model and requirements, and reported spending from 15% to 
25X of their time on supervision and evaluation. At least half commented 
further that the supervision and evaluation process was easier now, because 
they had so much data (anecdotal notes) in front of them when they sat down 
to confer with teachers, that the teachers were surprised and/or impressed. 
Two specific examples were also cited of work with initially reluctant/ 
uncooperative teachers who did not meet the district's instructional stan- 
dards. It was pointed out that the union supported the principal's work 
with these teachers and acceptable improvements have since occurred. 

At least half of the principals noted that finding time for their new 
role was a problem, and also indicfXed that the amount of paperwork in- 
volved for the monitoring/reporting process was an issue with principals. 
Two indicated that the district had responded to principals' requests to 
reduce the required number of conferences per month* Over two-thirds 
stated that they were now spending even more time with selected teachers 
who they felt needed assistance. Kone of the principals related any major 
problems at present with their staff development role. They cited the 
initial continuing assistance of SDT staff with teacher inservic.e, and 
elaborated on their own contributions to PRISM inservice and the emphasis 
they have placed on it. The general impression that came across, overall, 
was that they felt they were well trained and felt good about what they 
were doing. 

Impact of Supervision 

All of the principals felt that the PRISM inservices and the related 
supervisory observations have had a positive impact on teachers. They per- 
ceived that the majority of teachers generally viewed the program, posi- 
tively, accepted it, agreed that it had effected their teaching skills, and 
were aware that the PRISM observations were for improvement purposes only. 
Among the positive teacher-related outcomes cited were: teachers* in- 
creased sensitivity to students; sensitivity to their own strengths and 
I eeds regarding lesson design and implementation; increased awareness of 
effective teaching strategies; focus on the elements of lesson design; an 
increased sense of professionalism among high school staff; and development 
of a common language of instruction among all staff. Principals also 
perceived an ii"icrease in their rapport with teachers and increased support 
for teachers. 
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They attributed the effects of the program to the quality inservice 
materials and support made available by SDT sraff; che relevance of the 
inservice content; the sustained nature of inservice (three and one-'half 
years of PRISM input and review sessions); and the positively reinforcing, 
constructive and confidential nature of the classroom observations of 
teachers. A few principals noted that although most staff have bought into 
the program, "a few are still not tickled by it," They reported there was 
some variability among buildings in how it was introduced and in the degree 
to which it was emphasized. They felt that "the full word about the entire 
district package never reached some people." They concluded that teachers' 
acceptance had ^'a lot to do with the boss and with the f rust/respect that 
he/she developed," 

Impact of Evaluation 

The common perception among principals was that "teachers take the 
DEBE-333 evaluation in somewhat of a low key, routine, pro forma fashion." 
Most felt it had little impact on teachers' behavior except ir. those cases 
where unsatisfactory ratings were given* It is done to satisfy the state 
requirement « Most principals also reported spending relatively little time 
on the DEBE-333. Three of the principals stated, however, that they spent 
30 to 45 minutes on each evaluation. They add individualized reinforcing 
notes or letters to each teacher evaluation pointing out the positive 
attributes, contributions and areas of growth of each teacher based on 
specific occurrences over the previous year* 

There is also a growing perception among principals that the pro forma 
nature of evaluation is changing. Comments heard were: "The superinten- 
dent has shown he believes in giving unsatisfactory ratings. He's modeled 
It." "The super intendiant stated that there are unsatisfactory teachers in 
the district. Now, let's do something about it. Unsatisf actories were 
unheard of a few years age; now principals are doing it more." 

These perceptions were verified by district central office staff who 
reported that there has been an upward trend in the district in the number 
of unsatisfactory teacher evaluations given. In previous years, 3 or 4 
unsatisf actories per year were the average. In the past two years, 12 to 
14 unsatisfactory ratings were given each year. Last year (1983-84) , 4 of 
the 12 constituted the second unsatisfactory in a row that the teachers had 
received. Two of these teachers resigned short of a hearing and two others 
chose a hearing. In any event, the district's pri^,ary focus will continue 
to be to promote instructional improvement through staff development and 
supervision in order to make good teachers even better. Unsatisf actories 
are only to be given to those staff who do not respond to the extensive 
training and supervision* and fail to Jieasure up to the district's standards 
for classroom performance after sustained periods of assistance. The dis- 
trict's new PRISM-modified DEBE-333 evaluation foxin should help to further 
clarify the conceptual relationship between teacher supervision and eval- 
uation. 
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Perceived Strengths 

When asked to coiranent on the strengths of the program the principals 
cited the following: 

• quality training and materials 

• number of required observations 

• conmion language was established 

• strong superintendent and board commitment 

• adequate resources were allocated 

• clear district goals 

• program was introduced slowly 

• everyone had a coach 

• teachers liked getting positive feedback. PRISM was 
presented in a non-threatening way 

• positive feedback (to principals) from the coaches was very 
important 

• generally better instructional skills by teachers 

• can now professionally evaluate instruction 

• increased student achievement 

• teachers had significant impact on the design of the model 
high school teacher center 

• the eight-^week program at the teacher center 

• the weekly meetings of core staff representatives (PRISM SDT, and 
model high school directors) with the superintendent 

• support from the SDT and others for the teachers^ PRISM inservice. 

Some of their specific, more detailed comments include the following. 
"It's been a very positive note. We see for once, the whole city moving in 
one direction. People are recognizing it's been good for the system." 
*'I've seen changes in attitude— more professionalism. You don't see 
teachers sitting around reading a paper as in the past." "The quality of 
training and SDT support was excellent — they are there when you need them." 
"There has to be accountability. Principals and teachers can't think 'this 
too shall pass.' The superintendent has sec that tone very well-*-it won't 
pass!" 

Perceived Problems 

The predominant problem mentioned by almost all of the principals was 
finding time to do the required teacher observations and conferences, A 
few mentioned, however, that their new role has forced them to become 
better managers both of their time and in delegating some of their respon- 
sibilities. Other problems, each reported by at least two principals. 



88 



BEST COPY AVAILABLE 



were overcoming teachers' initial mistrust of the Intent of the program and 
the perception of some that it wasn't anything new; the duplication of 
effort involved in the "burdensome paperwork" and reports associated with 
the PRISM monitoring system; the fact that the program is an add-on and is 
seen as more work with less and less; difficulties encountered in doing 
adequate observations, in some cases, due to a lack of subject matter know- 
ledge; and a possible district overload regarding PRISM related tasks» 

Additional problems, each reported by at least one of the principals 
in the group included the following* 

• "Perhaps the district is trying to do too much in too short a 
period, and the SDT staff Is spread too thin." 

• "Rumors among teachers that the PRISM quiz was to be used for 
evaluation." 

• "The labeling of A-E conferences was confused with grading by 
some teachers," 

• "The perception by principals that some type of formal evaluation 
of the administrators was to be carried out by the SDT." 

• "The lack of a solid plan for 'visiting teacher' follow-up after 
their model high school teacher center experience." 

Perceived needs 

Each of the following were perceived as a need by at least one 
principal: 

• to expand training beyond PRISM to other research areas 

• to drop the observation quotas and allow principals flexibility 
regarding the number of observations and type of conferences 
conducted ("Let us budget our time. We We gotten the message by 
now/') 

• to have central office staff acquire a "better reality base of 
the many things going on in our buildings — overload" 

• to train substitute teachers in PRISM 

• to work with the middle school teachers in a manner similar to the 
high school teacher center to standardize PRISM (training) 
across schools 

• to receive more training In handling discipline 

• to Improve further the principals' networking process 

• to use one or two days before school starts for PRISM review 

• to provide more room on the DEBE-333 for providing outstanding 
ratings and/or recognition to deserving teachers 

• to provide for rotation of RTs and CRTs in the high school teacher 
center to cut down on burn-out and establish equity between the 
roles* 
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Teachers ' 



Perceptions 



Fifty teachers were interviewed regarding their perceptions of super- 
vision and evaluation in the Pittsburgh school district. Nineteen taught 
in five different elementary schools, 20 in three middle schools, and 11 in 
two high schools. 

Teacher Supervision and Teacher Evaluation 

Teachers generally made a distinction between the supervision (help- 
ing) and evaluation (rating) processes (i.e,, between PRISK and the DEBE- 
333), but said that they were related to one another, at le£6t partially, 
because information that principals obtain through PRISM classroom obser- 
vations are used in the end-of-the-year evaluations. Some teachers seemed 
to consider the relationship appropriate. For example, one teacher said, 
"It's a positive connection." Another said, "I would hope that PRISM would 
be part of it (the end-of-the-year evaluation) but not total .. .other things 
should be considered also," Others felt that administrators had misled 
them Initially by saying that PRISM would not affect teachers* evaluation 
ratings. 

Teachers' descriptions of the supervision process (i.e., announced or 
unannounced observation, focus on concepts included in PRISM training, 
anecdotal note-taking, and post conferences) were similar to admini- 
strators' descriptions. Most teachers said they were observed 4-6 times 
during 1984-85, 

Satisfaction With Input 

Only one teacher who was interviewed had direct input into the deve- 
lopment of the PRISM process; that teacher was on the committee that de- 
signed the high school teacher center. Most teachers were not dissatisfied 
over their own lack of involvement; they either believed that the teachers' 
union was involved or received some satisfaction from being among the first 
participants. A few teachers believed that PRISM was forced on them by the 
district administration ("I think it's Dr, Wallace's baby.") and/or would 
eventually fade away ("This, too, shall pass."), 

Utility/Impact of PRISM/ Supervision 

Nearly all teachers felt that PRISM had helped improve their teaching 
performance. Approximately half named specific changes in their teaching 
behavior sujh as stating the objective of a lesson or writing it on the 
chalkboard, using different questioning techniques, checking for under- 
standing, or allowing students sufficient wait-time. Some teachers re- 
ported becoming more aware of what they were doing and more deliberate and 
conscientious ("It brings to a conscious level all the things an effective 
teacher does naturally," "I question myself more and reflect on what I'm 
doing and see how I could improve."). Several teachers said that receiving 



90 



no 



ERIC 



BEST COPY AVAILABLE 




positive feedback (after being observed) was rewarding or helped build 
confidence; also, they viewed PRISM training as a refresher that helped 
them get out of a rut ("We gained confidence in learning that what we were 
doing was right." "I've gotten a lot out of it because someone comes in 
and says you are doing something right.") • Hany teachers reported that a 
ma.lor impact of PRISM was that they learned new terminology. A good number 
found that useful, like the teachers quoted above who became more conscious 
of what they did or gained satisfaction from learning that what they were 
doing was right. Others appreciated the terminology because it established 
a common language and common expect ions. Only two or three teachers viewed 
it as a wast6 of tim^. 

Teachers were also asked, "What percent of the total knowledge and 
skills, independent of content, that you currently employ in teaching would 
you attribute to (1) your college/university training, including practice 
teaching; (2) your experience over the years and/or interaction with peers; 
and (3) your PRISM experience?" Most teachers (80%) attributed more than 
half of their teaching skills to their experience over the years and 
interaction with peers. Forty percent of the teachers attributed 10%-15% 
of their knowledge ax\d skills to PRISM, 20% attributed 20% to PRISM, and 
40% attributed 25%-35% to PRISM. Two teachers attributed 40% and 60%, 
respectively, to PRISM. 

Utility/ Impact of Evaluation 

Teachers did not believe that the DEBE-333 annual evaluation helped 
improve their performance. They said that teachers are rated only "satis- 
factory" or "unsatisfactory," Most teachers receive the former. No other 
information is provided — for example, ratings on specific criteria or 
descriptions of strengths or areas that need improvement ("It Is not an 
incentive." "It's like getting a C average."). 

System Fairness 

Due to time constraints, the interview question about the fairness of 
the evaluation system was asked of only a few teacher? . They expressed 
concern about fairness, saying that ratings depend on the principal (e.g., 
a teacher was rated "satisfactory" in one school but "unsatisfactory" in 
another), that one teacher who was "just a body" received a satisfactory 
ratings and that the DEBE-333 is not fair to the school system because 
principals cannot give an unsatisfactory rating unless he or she has 
"mountains" of data to support it. 

S trengths 

Most teachers viewed PiilISM as a definite improvement over the DEBE-333 
observations conducted in the past. They said that PRISM was more objec- 
tive and that a common language and expectations had been established. It 
has also provided a framework for evaluating teachers (several used the 
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term "evaluation" here) that can be used across the district, with teachers 
of different subject areas ("Everybody is playing the game by the same set 
of rules. When the administrator comes in you know what he is looking 
for." "It's a lot more objective. It makes your evaluation more mean- 
ingful." "I should be able to go to any school in this system and know 
what any principal or supervisor would be looking for,,. We've needed some 
standardization/*) . 

Another PRISM strength that teachers described was that it helps 
identify teachers' weaknesses and provides direction for remediation. As 
mentioned above, it provides a framework with specific criteria. Also, 
principals and teachers now work together more to improve instructional 
skills. Before PRISM was implemented, there was little communication re- 
garding classroom observations; now there is more follow-up ("It can 
identify weaknesses and you get some constructive help." "It turns your 
needs into strengths." "We think the principal is working with us to en- 
hance learning in the school."). Other PRISM strengths that teachers 
mentioned included: the extensive training ♦ the opportunity to learn and 
interact with peers during training, and the quality of interaction between 
administrators and staff. 

Needs/Problems 

The needs/problems that teachers mentioned most frequently referred to 
administrators' roles in PRISM. First, they mentioned problems related to 
the system's dependence on administrators. They said that the implemen- 
tation of PRISM varies among schools, depending on principals' attitudes 
toward it and their relationship with the staff. Some principals have in- 
troduced PRISM more thoroughly and effectively than others. Second, 
teachers mentioned problems that have arisen from the quotas that require 
principals to conduct a specified number of observations. Because of the 
quotas, teachers are sometimes observed too frequently, for example, when 
they teach in a small building or when several different observers who are 
trying to meet their quotas converge on a teacher without coordinating 
their visits. Also, as quota deadlines approach principals sometimes are 
not available vhen needed. 

Other needs or problems — each desc" ■'bed by only one or two teachers — 
included; 

• lack of reinforcement or ongoing training 

• differences in teachers' training needs 

• differences among teachers in the frequency of observations 

• inadequate communication with teachers early on 
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• insufficient information provided to teachers who assisted with 
training 

• use of PRISM to document cases against teachers* 



District Administrative Staff's Overall Reflections on 
Program Design, Operation, and Impact 

This section includes comments derived from interviews with the 
director of the PRISM SDT, the five SDT staff members, the director of the 
high school teacher center, and the six district assistant superintendents. 

Perceived Strengths 

When asked about the strengths of the district teacher supervision/ 
evaluation system (PRISM) administrative staff cited the following* 

• "The common language of instruction » the coaching, and the absence 
of 'official observation write-ups' of teachers and administrators 
PRISM-related performance are key strengths," 

• "The use of positive reinforcement in the conferences with teachers 
in the first year or two was a critical factor in building trust," 

• "The large number of the above observations/conferences also built 
trust," 

• ^'The use of a R&D-based staff development model based on Bruce 
Joyce's writings," 

• "The development of the SDT ('We consciously made a decision to use 
our own staff and not put a guru up front,')," 

• "The group planning process which included union representation 
('The union agreed to the choices of their members on the 
committee, ') 

• "Having the administrators teach PRISM to their staffs increased 
their knowledge of PRISM," 

• "The SDT has become an integrating force across the district — 
reaching across many of the districts priority programs. Most new 
district committees starting ap want a SDT representative on them," 

• "The four clusters of support groups for elementary level princi- 
pals," 

• "Completion of the formal connection between PRISM and the 
DEBE-333," 
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• "The systematic monitoring and/or tracking of the required PRISM 
observations/conferences, and principals* growth/progress by the 
SDT helped to promote and verify the standardization of proce- 
dures/' 

• "The fact that central office staff received the training added 
credibility to the process in the eyes of principals/' 

• "Our use of pi'incipals (SDT members) to teach other principals in 
order to capitalize on peer influence*" 

• "The fact that we started with the secondary school; something that 
most districts have hesitated to do." 

• "PRISM has helped teachers to improve." 

• "TVie teachers' union now feels we s^z ioing teacher supervision/ 
evaluation more objectively and that the principals are better 
trained to do the jjb." 

The administrative staff also offered the following perspective on 
PRISM. "We see PRISM as a means to a larger end. It's not the 'end-all or 
be-all' of teaching. It's a foundation. It^s an oversimplified approach 
to get all staff on the same footing, (common language). We're using PRISM 
as an overlay for other models of teaching. For example, we'll start deve- 
loping questioning and discussion skills in September 1985. In essence, 
the first cut was the generic PRISM model* Now we are looking at ways to 
make the model 'content or subject area' specific. The district is be- 
ginning to examine ways to coordinate the supervisory efforts of principals 
and subject matter specialists." 

They added that, "Initially, the board established the staff develop- 
ment team for five years — until the new evaluation system was in place. 
Now the superintendent wants to continue the team to help maintain, refine 
and expand the various PRISM efforts, and to introduce new training. We 
anticipate that by the end of the 1985-86 school year we will turn the 
PRISM I, Stage 1 training responsibility over to supervisors (after five 
years). The SDT will then move on to new content and training areas and 
begin new training of staff*" 

Perceived Problems 

When asked about the problems encountered in implementing PRISM, admin- 
istrative staff reported these two. "The biggest problem was in training 
ourselves (SDT), developing materials and selling the program to school 
staff. We need to continually grow." "Having administrators teach PRISM 
to their own staffs at the end of the first year was also a problem and was 
the weakest link in the program at that time. There was considerable 
variability in the training among schools. In the end, though, the de- 
cision paid off because the principals learned the model with the help of 
SDT staff. After the end of the second year we made the requirement 
optional." 
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Other problems involved: (1) selling the model to teachers and con- 
vincing them th;it it was not going to require them to all teach in a lock 
step fashion » rather its a decision model and frees teachers to make more 
decisions in a more informed rational way; (2) the complaints of principals 
regarding time — their PRISM role is part of their job description, however, 
and they all managed to realign their priorities; and (3) stresses in the 
demanding CRT role in the model high school teacher center which resulted 
in 10 CRTs leaving this year (burn-out). Changes are being considered in 
modifying both the CRT selection process and the labor intensity of the 
role requiremc-nt. 

Changes Made or Things They Might Do Differently 

In the course of implementing the project, district staff made the 
following changes or adjustments. They lowered the requirement of 12 obser- 
vations per month by principals to 6 in the third year of the program. 
They moved from requiring A, B, and C-type conferences (10-15 minute 
observations and conferences) to refinement conferences (20-30 plus minute 
observations and equally lengthy conferences) in the second or third year 
of the program. They reduced the summer training from ten to five days 
after the first suumier, primarily at the request of the principals. They 
continually modified the training materials tased on participants' feedback 
and attempted to individualize PRISM training due to differences in 
teachers' entering knowledge of the program. 

They would also relabel the A, B, C conferences, and "name" them if 
they were to do it over, due to the confusion with grading that the "A, B, 
C," labels caused. In retrospect, they would also have brought Ernie 
Stachowski in earlier to help SDT staff plan the program. 

Anticipated experience-based changes or improvements in the program 
include: (1) developing a stronger teacher follow-up plan for the proposed 
Pittsburgh Elementary Model School (PEMS) , and more directly involving 
principals in the follow-up; (2) taking more care in selecting CRTs for the 
Elementary Model School by providing PRISM and some CRT role training 
before the selection; (3) alternating the roles of RTs and CRTs in the PEMS 
to provide equity in status, opportunity, and down time atiiong staff; 
(4) taking a semester (fall 1985) to gear up the PEMS (training/selecting 
RTs, CRTs and replacement teachers) before commencing the training of 
elementary visiting teachers and principals; and (5) increased evaluation 
efforts regarding the outcome effects of PRISM, on both teacher performance 
and student achievement. 

Assistance They Could Have Used 

Administrative staff reported that they could have used and would like 
to have used in-stal'j assistance with program planning and SDT staff 
training, if any of the educational agencies in the region had the 
capability. However, they did not. District staff also did not perceive 
the Pennsylvania Department of !iducation as a viable source of ^ssistance. 
They perceived that they had no choice but to go out of state for qualified 
consultant assistance with PRISM. 
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Advice to Others 



When asked what advice they had for other districts considering the 
development of similar teacher supervision programs, administrative staff 
offered the following suggestions and cautions. 

• A district has to commit to a goal, a plan, and a program that have 
the support of the board, superintendent, and the central office* 
That is, a district needs a vision of where it wants to go and how 
it plans to get there • 

• Adequate monetary resources have to be provided. 

• The decision hierarchy needs to be flattened to facilitate 
program planning and implementation ("We get to set the agenda for 
the weekly meeting with the superintendent. Initially we didn't 
feel secure enough to make certain decisions on our own. Some 
decisions need to be made immediately.") . 

• Involve teachers in the planning. 

• Listen to the teachers* input regarding their decisions about their 
needs and desires (in the context of the goal)* Whatever their 
input decisions are, the district has the right to shape them but 
must keep faith wi:h the spirit and intent of teachers' decisions. 

• Choose an instructional/supervisory model — like the "instructional 
committee did." 

• Use in-house trainers and train the trainers. 

• Provide coaching to all staff whose skills you are trying to improve. 
Deal with the positives first. Give staff positive reinforcement 
regarding what they are doing right. 

• Go through all of the staff training stages suggested by Bruce 
Joyce — provide time for modeling, guided practice, feedback, and so 
forth* 

• Recognize that change is part of Che process. As people use the 
model they look for different ways it can be used* There is a 
need for flexibility. 

• Go to the research for initial program content, training protocols 
and evaluation. 



Summary of Findings and Discussion of Implications/Issues 

Summary of Findings 

The efforts of the Pittsburgh Public School District to improve the 
quality of instructional supervision, teacher evaluation, teachers' in- 
struction, and the overall instructional and educational leadership skills 
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of principals appear » in large part» to be meeting with considerable 
success. The improvement efforts over the past four years can be 
characterized as well planned ^ designed, imp lenient ed , and evaluated* The 
districts' efforts in the above regard are also noteworthy for their 
clarity of purpose; comprehensive, intensive and sustained staff 
development; R&D base; monitoring and follow- through; and even for the 
superintendent's modeling behavior* 

Some of the specific factors or strengths of the PRISM system and the 
related procedures employed in implementing the system that appear to have 
contributed to the district's success can be listed as follows* 

• In the planning and design phases of the various PRISM Initiatives 
the district engaged in needs assessment and participative planning, 
and used R4D-based information to guide its decision-making. The 
district selected a single instructional model (for a foundation) 
and a R&D-based staff development model. Trainers homophylous with 
the Initial intended training audience (principals) were also 
selected and provided with intensive training. Nationally recog- 
nized consultants were used for planning and start-up assistance. 

A long-term timeline (5 years to start) and an incremental imple- 
mentation plan were established* Systematic formative evaluation 
was also planned* Finally, the district made the decision, up 
front > to conceptually and procedurally connect supervision and 
evaluation after a sufficiently long phase-in period. 

• Program strengths involved during staff development and program 
Implementation, related to principals* PRISM roles, include: 
quality training by SDT staff in Hunter's essential elements of 
Instruction and clinical supervision; extensive modeling, practice 
and coaching during training; institution of multi-year, long term 
SDT follow-up coaching of principals; formation of long-term sup- 
port networks for principals; the requirement that principals teach 
PRISM I to their own staff (with SDT support); the requirement that 
principals conduct a set number of observations/conferences per 
month; the requirement that principals prepare annual staff deve- 
lopment plans; the extensive monthly and semi-annual monitoring of 
principals' Implementation of both the observations/conferences and 
staff development; the annual provision of supplemental PRISM II 
summer training for principals in related instructional leadership 
topics/skills; the abstinence from evaluating formally the quality 
of principals' performance of their PRISM duties in the phase-in 
years of the program; the sustained positive aupport and rein- 
forcement provided principals during PRISM training and implemen- 
tation; and provision of in-depth orientation/training experiences 
for principals at the model high school teacher center* 
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• Prograro strengths involved during staff development and program 
Implementation related to teachers* role in PRI' .i include: system- 
atic PRISM I training and review, over a four year period, by 
principals (and SDT staff) with quality materials; extensive class- 
room observing/conferring of teachers (3-10 times annually); absti- 
nence from maintaining "official" records on teachers' PRISM per- 
formance iuring the four-year PRISM phase*-in period; the focus on 
providing maximum positive reinforcement to teachers during the 
first two years of their training; the emphasis on refining and 
adding to teachers* skills in the latter half of PRISM phase-in, 
especially those most in need; the provision of an in-depth^ eight- 
week, clinical teaching experience for secondary school staff at 
the model teacher center; the follow-up prograti: Instituted to 
assist teachers after their teacher center experience; and the four- 
year phase-in period before teachers were formally held accountable 
for PRISM skills on their annual DEBE-333 evaluation. 

The mammoth improvement effort undertaken in Pittsburgh was not with- 
out its share of problems. Some of the more apparent problems included: 
principals' and teachers' initial resistance to the program; unevenness in 
the quality of PRISM I inservice for teachers provided by school principals; 
principals* problems in finding time for the observing/ conferring and 
related paperwork required for PRISM; burn-out and role-conflict problems 
between the clinical resident and resident teachers at the high school 
teacher center; perceptions among some staff of "PRISM overkill" regarding 
training and district emphasis; problems in the design and effective 
implementation of the follow-up program for secondary teachers after their 
visit to the high school teacher center; and problems reported by some 
principals in reaching a ceiling or impass, so to speak, in the 
effectiveness of their supervisory efforts with teachers, due to 
constraints imposed by their lack of knowledge of the subject matter (i»e., 
the generic Hunter model needs to be further developed and/or tailored to 
take into consideration inherent differences in subject matter and varied 
objectives being pursued within the subject matter)* 

Implicat ions /Issues 

The experiences of Pittsburgh school system staff regarding their 
efforts to improve the district's teacher supervision /evaluation system^ 
and the quality of instruction and instructional leadership, are rich in 
hints to other districts interested in implementing similar improvement/ 
implementing efforts. Many of these hints are implicit in the findings 
reported above. Among the more salient implications are the following. 

• If principals are truly to become Instructional leaders they must 
be provided the training and then be put in the position of super- 
vising and training staff. To the degree that principals acquire 
the capability to train their own staff (with appropriate support), 
they will, in turn, then be preceived by their staff as having the 



98 



9S 



ERIC 



BEST COPY AVAILABLE 



skills and entitlement to supervise them and concomitantly to add 
to their instructional skills. They will also not truly learn and 
become confident in their skills if they sit back and watch others 
do the training* Learning is doing. Doing, during ones' own 
training is a necessary condition for learning a skill ♦ It may 
not, however, be sufficient enought to enable one to teach others. 
The experience of teaching others may be a necessary condition re- 
quired to enable one to become an effective supervisor. At the 
ideal level, there is a close correspondence between teaching and 
supervising* 

• Debates exist over whether or not teacher supervision and eval- 
uation can be conducted effectively by the same school personnel* 
Some (McGreal, 1984) recommend that they be separate functions 
conducted by separate staff* The financial and staffing realities 
of most school districts preclude this division of labor. The 
Pittsburgh model appears to offer a viable alternative* The debate 
is based on factors of credibility/expertise and bias/trnst. The 
compromise to the issue, as reflected in Pittsburgh's model, is 
that well trained principals can effectively perform both roles 
with increased credibility and teacher acceptance if they are suf- 
ficiently separated in time and purpose. In Pittsburgh the dis- 
trict waited four years (and offered four years inservdre) before 
"formally" holding teachers accountable (evaluation) for the 
teaching skills imparted in PRISM staff development and related 
supervision. The absence of formal written reports on teachers' 
performance during the extended teacher practice/feedback phase-in 
period of the program, and the initi:Ll emphasis on the positive 
(what teachers were doing right), appear to be key components of 
the model. Thus, given the right mix of appropriate training and 
sustained support, it appears that a principal can be perceived as 
both an "entitled supervisor" and an "entitled evaluator" if the 
district does not attempt to achieve this end overnight. Time for 
intensive practice and feedback by both principals and teachers, in 
a sheltered, nurturing environment, is essential to both skill 
development and staff acceptance. 

• Coaching is a critical ingredient in the Pittsburgti model. The 
coaching cylce involves observations, feedback, and modeling. In 
Pittsburgh, principals coach teachers, SDT staff coach principals, 
and consultants coach SDT staff. The learning and transfer of 
skills requires on-the-job practice and feedback. Coaching, would 
appear to be a necessary component of a successful program from 
several perspectives: feedback or knowledge of results, modeling, 
motivation or reinforcement, monitoring, and interpersonal 
interaction/recognition* . 

• Finally, the Pittsburgh staff would be the first to state that the 
Hunter model is not the end-all, be-all of educational improvement. 
It is a robust, practical, R&D-based foundation to get staff on a 
conmion footing or frame of reference regarding baseline effective 
instruction upon which to begin to build staff communication and 
skills toward more subject-matter specific instructional skills. 
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CASE STUDY hi TAMAQUA AREA SCHOOL DISTRICT 



Background Data 



Schools 



Students 



• 3 Elementary (K--6) 

• 1 Junior High (7-8) 

• 1 High School (9-12) 



• 2,457 students 

• 99.6% white 

• Student population 



declining 



Staff 



• Superintendent 

• Assistant Superintendent 

• 1 Elementary School Principal 

• i High School Principal and 2 Assistant Principals 

• 1 Assistant Principal Responsible for the Junior High 

• 64 Elementary School Teachers 

• 76 Secondary School Teachers 

Average Per-Pupil Expenditure 

• §1,752 for elementary students; $2,197 for secondary students 
Teacher Supervision/Evaluation 

• Evolving observation system (no official label), begun in 1978-1979 

• DEBE-333 



The Tamaqufi Area School District, located in Schuylkill County, 
includes four tovnships as well as the borough of Tamaqua* The district's 
18,000 residents live in an area that is more than 120 square miles and 
includes farmland, seven small communities, the borough, a major river (the 
Schuylkill) and several creeks, mountains, and valleys. Many people are 
employed in farming, industries, mining, and service jobs. School adminis- 
trators have characterized residents as conservative and traditional. 



Community 



lOI 
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Program Initiation, Planning, and Start-up 



Initiation 

The teacher supervision/evaluation system was Initiated by the current 
superintendent In 1978-79 when he assumed chat position. At that time, the 
district had no formal procedures for teacher supervision and evaluation* 
The superintendent Introduced the system to eliminate that deficiency* 

Planning and Start-up 

Based upon his own teaching experiences, his work at Temple Uni- 
versity, participation in PDE and Intermediate Unit training sessions on 
Instruction and supervision, and acquaintance with ASCD training materials 
on teacher supervision, the superintendent identified seven components of a 
successful lesson and then trained administrators to use them during class- 
room observations. The seven components are: (1) start on time; (2) pro- 
vide for readiness and Insure continuity with the previous day's lesson; 
(3) state objectives, expectations and purpose; (4) use appropriate teach- 
ing strategies which include student Involvement; (5) bring the lesson to 
closure; (6) evaluate students' learning; and (7) provide homework. The 
initial training of administrators consisted primarily of tutorial sessions 
conducted by the superintendent at regularly scheduled staff meetings* 

AdmlRlstrators had input into the system, but others did not* 
Teachers were not Involved in program development, either as individuals or 
through the education association. Generally, however, teachers did not 
object to the system* 

Staff Development; Content, Process and Evaluation 

Administrators: Content and Process 

Training in the supervision system generally has not been extensive or 
formalized* Various administrators have been exposed to a variety of 
supervision-related workshop sessions in recent years* However, not all 
administrators have participated in all sessions* Training was presented 
through the following sources: 

• initial system training by the superintendent 

• PDE Executive Academies 

• recent lU 29 program on Teacher Supervision and Evaluation 

• ASCD videotapes (Dick Manatt) 

• ASCD conference 

• lU 29 workshop conducted by Tom Perslng of the Upper Perklomen 
School District 
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• superintendent's review of observation reports and subsequent 
conferences with administrators 

• observation (during internships) of vice principals' classroom 
observations /post-conferences and subsequent critiques conducted 
by the" superintendent and high school principal 

• discussions during bi-weekly administrative staff meetings* 

Teachers: Content and Process 

The district has not attempted to provide systematic training for 
teachers in the supervision process* Teachers have learned about it 
through: 

• individual review of their observation reports 

• post'-observation conferences with principals 

• presentations by principals at faculty meetings 

• descriptions of observation criteria distributed by some 
principals* 

Also, four teachers attended the lU 29 workshop conducted by Tom Persing of 
the Upper Perkiomen School District. 



Implementation of Supervision/Evaluation System 
Philosophy /Policy 

District administrators view the observation system as a more useful 
form of evaluation than the DEBE-333, one that can be used for management 
and supervision. They believe that it helps administrators establish 
expecta^ .ons of teachers » acknowledge good teaching* assist individual 
teachers, and identify common staff development needs. The DEBE-333 is 
used to satisfy state requirements anH is the official document used in 
personnel decisions. However, the observation system is the operative 
evaluation tool. Administrators use knowledge obtained during observation 
visits when completing the DEBE-333» Observation reports include a 
statement that the teacher either is or is not satisfactory. The report 
format contains a lengthy section labeled "evaluation." Administrators and 
teachers use the term evaluation when discussing the observation system. 

Requirements 

Generally, it is expected that non-tenured teachers will be observed 
once each semester, elementary teachers once every two years, and secondary 
teachers every year. Observers, who usually do not announce their visits 
in advance, remain in classrooms during an entire class period. They take 
anecdotal records; prepare written reports that include the timed record of 
events, a statement about teacher preparation, an evaluation of the lesson 
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observed^ and recoinmendations; and conduct post-conferences. Alternatively, 
some observers hold post-conferences before putting their notes in type- 
written form. Later, a description of the post-conference Is added to the 
report and the teacher is then asked to sign it, Pre-conferences are held 
only as needed — usually with non-tenured teachers. The district does not 
have formal standardized observation criteria. Each observer uses a vari- 
ation of the "successful lesson components" assembled by the superinten- 
dent. Two or three administrators have defined the components in writing 
and shared them with their teaching staff. The operational definitions, 
however, tend to vary in format, emphasis, detail, and content, 

DEBE-333 evaluation reports are completed annually on the basis of 
information obtained during classroom observations and informal inter- 
action. The reports are shared with teachers only if a problem is identi- 
fied. Teachers who receive unsatisfactory ratings are observed again with- 
in 4-6 weeks. 

Organization and Process 

Supervision and evaluation responsibilities are shared by the superin- 
tendent, assistant superii^tendent, two principals, and three assistant 
principals. Most elementary school teachers are observed by the principal, 
although the superintendent observes some and the assistant superintendent 
observes Chapter 1 teachers, Suyjervis ion/evaluation responsibilities for 
secondary teachers are organized by department. The superintendent, as- 
sistant superintendent, secondary principal, and assistant principals are 
each assigned to one or more departments, (Management responsibilities are 
divided similarly; administrators serve as department chairs.) Conse- 
quently, some teachers are evaluated/ supervised by administrators from 
other buildings- 

Each secondary school administrator is expected to conduct a minimum 
of 10 observations per year. Although minimums have not been established 
for the other administrators (superintendent, assistant superintendent, 
elementary principal) , they reported conducting 15-20 observations each 
year. 

System Monitoring 

The superintendent monitors both the quantity and quality of obser- 
vations. His secretary maintains a record of the number of observations 
conducted by each administrator, and periodically sends them a reminder of 
the number remaining for the year. The superintendent alao maintains a 
record for each teacher that shows the dates of observations and names of 
the observers. 

The superintendent receives a copy of each observation report. He 
reviews the reports, makes notes on them, and then holds conferences with 
individual administrators. Generally, he is satisfied with the quality of 
the observations. Additional monitoring occurs when the secondary princi- 
pal, who receives copies of all high school teachers' observation reports, 
reviews them and gives informal feedback to observers. 
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Administrators' Percepfrions 

All school-level administrators (the elementary principal, the secon- 
dary principal, and the three assistant principals — one of whom serves as 
principal of the junior high school) were Interviewed to obtain their per- 
ceptions of the district's supervision/evaluation system. 

Relationship Between Teacher Supervision and Teacher Evaluation 

Administrators say that the district distinguishes between supervision 
and evaluation, and can point to two sets of procedures — the observation 
system and the DEBE-333. 

Utility of Training 

Only two administrators commented on the effectiveness of the 
training, saying that It was "good." However, all five expressed a need 
for additional training. The most frequently expressed need was for 
training In conferencing skills, mentioned by three administrators. One 
of those three named a related need—for skills In providing Information 
that will help Improve teaching. Other administrators desired training In 
the observation criteria. One would like to observe other administrators 
as they conduct observations. 

Feasibility of Implementation 

The major Implementation problem that administrators Identified was 
lack of time to conduct the observations. All five administrators said 
that they would like to devote more time to the process In order to observe 
teachers more frequently. They estimated that conducting an observation, 
writing a report, and conferencing with the teacher required three to four 
hours. 

A second Implementation problem, discussed by two administrators, was 
that teachers were not trained In the process. VSome have never even seen 
a list of the observation criteria. Consequently, those administrators 
believed many teachers have not accepted the observation system. 

Impact of Supervision 

Most administrators said that they felt that the observation system 
has helped teachers Improve their performance, although one questioned 
whether the effects were Immediate or long-term. Two administrators said 
that the system has helped communicate their expectations to teachers. One 
said that It has prompted currlcular changes such as discontinuing a 
reading series. 

Several administrators thought that teachers' reactions to the system 
were generally negative. One said that it was particularly "tough sledd- 
ing" at first, because teachers resented the time required for conferencing 
and perceived recommendations in observation reports as negative evaluation 
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statements. Other comments were that teachers feel that the system Is 
nothing nev or "pay lip service" to it because they know the administration 
expects it. 

Impact of Evaluation 

Administrators generally agreed that the DEBE-333 has little impact on 
teaching. In f act » unless there is a problem, teachers do not even see 
their DEBE-333 forms • 

Perceived Strengths 

Administrators considered the observation system's provision of an 
opportunity for communication between teachers and administrators its major 
strengths It allows them to work together to improve instruction and in- 
creases administrators' knowledge of classrooms. Other strenfjths are that 
it gives administrators criteria for observing teaching and guidelines for 
conducting observations , including taking detailed notes of classroom 
events. 

Perceived Problems and Needs 

Most of the problems /needs perceived by administrators have already 
been described. They include: 

• additional time for conducting observations 

• training for teachers in an instructional model and coinmunication 
of the system to them 

• more frequent observations 

• more pre~conf erencing 

• training for administrators, particularly in conferencing 
skills 

• finding a solution to problems arising from the current organiz- 
ation of supervision responsibilities. That organization requires 
administrators to observe teachers in subject areas they do not 
know, and observe teachers in buildings other than their own. 

Teachers' Perceptions 

Twenty-seven teachers were interviewed to elicit their perceptions of 
the district's supervision/evaluation system* Eight taught at the 
elementary level, ten at the junior high, and nine at the high school. 
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Teacher Supervision and Teacher Evaluation Procedures 

Teachers definitely perceived a di«;tinction between the observation 
system and the DEBE-333 evaluations, but their knowledge of both was vague 
and they viewed both as evaluation. No teachers had been trained in the 
observation process. Some had received copies of the ^bs.,rvation criteria, 
without definitions of them; some had heard oral presentations of them; 
and some had never been exposed to a complete list of them. Some teachers 
first learned of the criteria during post-observation conferences, deduced 
them from observation reports, or learned about them from other teachers. 
Some teachers were comfortable with the situation; others were not, saying 
that they were being held accountable for criteria that had not been com- 
municated to them* 

During interviews, teachers were able to name some observation cri- 
teria. There was overlap among them, but only one criterion — statement of 
objectives — was mentioned by more than one*-fourth (seven) of the inter- 
viewees. Slightly less than one-fourth named interaction with students, 
techniques /me thods , and closure. 

The frequency with which teachers reported being observed was similar 
to that indicated by administrators. Although a few were observed more or 
less often, most elementary teachers were observed every two years 3nd 
secondary teachers every year. The process was also similar: observation, 
written report, and post-conference. 

Most teachers were familiar with the DEBE-*333, but had not seen their 
own reports in recent years. Also, their perception was that virtually all 
teachers have been rated "Satisfactory Over the years, they said, 
several were not and subsequently left the district. 

Satisfaction With Input 

Only one teacher, a former local education association president, 
thought that teachers had been involved, through the association, in the 
development of the system. Another former association president said that 
teachers were not involved. Very few expressed dissatisfaction over this 
lack of involvement. Teachers in one department of the secondary school 
were given an opportunity to react to an observation checklist developed 
by the department chairperson, 

Uti^.ity/Impact of Supervision 

Most teachers (19 or more) said that the observation system does not 
help improve teaching performance. Those who gave a reason for their 
assessment said that observation reports do not identify any weaknesses, 
are not taken seriously because recommendations would be made even if a 
lesson were perfect, and are too infrequent. Six interviewees said that 
the system has helped slightly or that it has the potential to help. They 
said that administrators attempt to help teachers through constructive 
criticism, that the system is motivating or "helps keep you on your toes," 
or that the anecdotal records are useful — "like taking home movies," 
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Overall, although many teachers think the observation system has relatively 
little impact, they do feel it increases administrators' knowledge of their 
teaching, thus enabling them to support teachers against outside criticism. 

Utility/Impact of Evaluation 

No teachers reported that the DEBE-333 evaluations were useful ♦ Two 
said that teachers view them as a "joke/' 

System Pairness 

When asked whether they viewed the evaluation system as fair, teachers 
said that it did not distinguish between good and poor teachers. To some, 
this meant that all teachers were treated similarly and that the system was 
unthreatening and, therefore, fair. Several interviewees commented, how-- 
ever, that poor teachers should be given assistance or removed from the 
district; currently, very few are. 

Strengths 

The observation system strength that wa$ mentioned most frequently 
by teachers was the opportunity for interaction between teachers and 
administrators during the post-observation conference* This not only 
provided a forum for discussion, but it allowed administrators to attempt 
to help teachers and teachers to react to the observation report. 

Other strengths (mentioned by only one or two teachers) were the 
system^ s fairness and comprehensiveness, its non-threatening nature, and 
the training of the observer in the subject matter observed « 

Needs/Problems 

The need or problem identified most frequently (by at least one^third 
of the teachers interviewed) was the Infrequency of the observations. Some 
teachers said they thought the system could be more effective if admini- 
strators spent more time in classrooms; others said that administrators who 
knew more about them would be more able to defend them against board mem- 
bers' criticisms. Another problem area, mentioned by approximately six 
teachers, related to the organization of administrators^ observation re- 
sponsibilities. Teachers said they were being observed by administrators 
who lacked training in their subject area; whose offices were in another 
building and who did not know them on an informal, daily basis; or who had 
been out of the classroom too long. Other needs or problems identified by 
interviewees were that; 

• the system was too loosely structured and not sufficiently 
objective 

• the system was too tightly structured and formal 
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• teachers were not informed of the observation procedures 
^ sope teachers were observed more often than others 

• too little assistance was provided to some teachers • 



District Administrative Staff *s Overall Reflections on 
Program Design, Operation, and Impact 

Perceived Strengths 

One district administrator considered the amount of time spent on each 
observation as the system's major strength. Another said that the system 
*'is the best needs assessment that can be done." It has contributed toward 
the improvement of instruction — for example, more teachers now inform stu- 
dents of their expectations — although its impact might be greater if obser-- 
vations were conducted more frequently and if more teachers bought into it* 

Perceived Problems 

A major implementation problem was getting administrators out of 
their offices and into classrooms to conduct observations. This problem 
was alleviated by establishing a minimum of 10 observations per admini- 
strator per year. However, Itck of time is still a problem; one admini- 
strator thinks that each teacher should be observed several times every 
year* Another problem is that administrators still lack the skills needed 
to help teachers overcome weaknesses Identified through observations* In 
response to that problem^ the district is in the process of establishing a 
resource pool that administrators can use* 

Another set of problems appears to stem from teachers' lack of train- 
ing in the process* One administrator said that teachers do not understand 
the impact of the observation criteria on student learning* The other said 
that many teachers still have not accepted the observation system. 

Changes Made or Things They Might Do Differently 

Two changes that have been made were previously mentioned: the dist- 
rict established a minimum number of observations to be conducted by 
administrators; also, it is in the process of establishing a pool of re- 
sources to help teachers overcome weaknesses. The district also plans to 
obtain an ASCD film to use in training administrators. Asked what they 
would do differently if they had it to do over, one administrator said that 
the system is still evolving and that he would again let it evolve in the 
same way. The other said he would spend more time getting teachers to 
accept the system* 

Assistance They Could Have Used 

Both central office administrators expressed a need for resource 
people such as professors to help train administrators. Such resources are 
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more plentiful now but, according to one administrator, the "offerings were 
very shallow" in the late 1970 's. One also expressed a need for materials 
such as ASCD videotapes. 

Advice to Others 

The district central office administrators advised that others obtain 
input from administrators during system development and sell the system to 
the faculty. 

Summary of Findings and Discussion of Implications/ Issues 

Summary of Findings 

With little external assistance or expenditure of funds, the Tamaqua 
school district adopted an alternative teacher supervision/evaluation 
system that has similarities to much more expensive systems used in other 
Pennsylvania districts* Basic elements of those systems include classroom 
observations by administrators who make anecdotal records and use criteria 
based on nation ally -known research and theoretical knowledge on the process 
of instruction, written reports rather than checklists » and post-obser- 
vation conferences between the administrator and teacher. 

Administrators and teachers identified several strengths of the obser- 
vation system. It provides an opportunity for instructionally-related 
interaction between administrators and individual teachers, allowing them 
to work together to improve teaching. It establishes expectations of 
teachers and can be used to assist individual teachers • The system pro- 
vides guidelines for administrators to use in conducting observations and 
increases their knowledge of classroom teaching. 

The superintendent has demonstrated commitment to the observation 
system through his personal involvement in it. He has provided tutorial 
assistance and other training for administrators. He monitors the system 
carefully. He conducts observations himself* 

The district has responded to some of the system problems that have 
been identified* For example, in an effort to increase the number of obser- 
vations conducted, they established minimum numbers of observations that 
administrators are expected to conduct. Also, they are assembling a pool 
of resources to use in correcting teachers' weaknesses* 

However, Tamaqua 's observation system has several weaknesses that have 
not been addressed* The major weakness, which many administrators and 
teachers described as a serious obstacle, is teachers* lack of knowledge of 
the observation criteria. As mentioned earlier, teachers have not been 
trained in the System. Many do not even have a list of the criteria* 
Consequently, many teachers du not know what the district expects of them. 
Even when they are aware of a criterion, they do not have a clear under- 
standing of v'mt it means, how it affects student learning, or how they 
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should enact it, A correlary of this is that many teachers have not ac- 
cepted the observation system. 

Other problems that administrators or teachers mentioned included the 
following. 

• Observations are too infrequent. They would be more useful if 
administrators visited classrooms often and were more familiar with 
them. Related to this, administrators report that they do not have 
enough time to devote to the observation system. 

• Administrators need additional training in conferencing skills and 
in how to help teachers overcome weaknesses . 

• The style of observation reports makes it difficult to distinguish 
between good and poor teaching. Some teachers think that reports 
would include recommendations for improvements even if the observed 
lessons were perfect. 

Implications /Issues 

Seemingly, Tamaqua has demonstrated that alternative teacher super- 
vision/evaluation systems can be introduced at relatively low cost and with 
relatively few other resources such as external assistance or training for 
teachers. However, the fact that teachers have not been trained appears to 
have limited the system's effectiveness severly. Many teachers do not 
understand the criteria, know how or why to apply them, or accept the 
system as a viable framework for improving <;tudent learning. 

Most districts face constraints that limit the amount of training Chat 
can be provided to teachers. Many of those constraints are financial. 
Others, like Tamaqua's, are political: the community will object to having 
teachers removed from classrooms to participate in training. 

However, events in Tamaqua suggest that if districts want to introduce 
new teacher supervision/evaluation systems, but cannot provide extensive 
training for teachers, they should at least attempt to provide some 
training. Teachers need to know what Che observation criteria are and to 
understand them sufficiently to know how and why to apply them. While a 
week or more of training with an opportunity for practice and feedback may 
be desirable, alternative approaches might include: 

• printed lists of criteria, with clear, usable definitions 

• inservice sessions 

• presentations /discussions during faculty meetings 

• credit for participating in relevant courses or workshops 

• tutoring during pre- and post-observation conferences. 
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CASE STUDY 5: UPPER PERKIOMEN SCHOOL DISTRICT 



Background Data 



Schools 



Students 



• 3 Elementary (K-4) 

• 1 Middle (5-8) 

• 1 High School (9-12) 



• 2.980 students 

• 98% white 

• Student population 
stable 



Staff 



• Superintendent 

• Assistant Superintendent 

• Supervisor of Curriculum, Instruction, and Personnel 

• Business Manager 

• 2 Elementary School Principals 

• 2 Middle School Principals 

• 1 High School Principal and 2 Assistant Principals 

• 85 Secondary School Teachers 

• 79 Elementary School Teachers 

(65% of staff - 10+ years of experience) 

Average Per-Pupil Expenditure 

• S3, 392 

Teacher Supervision/Evaluation 



• Staff development program, Utility-Based Professional Staff 

Development (UPSD) , initiated in 1978-79, is based on the work of 
Hunter and others* 



• Staff select from five alternative modes of supervision (e«g., 
supervision by the principal or a colleague) , One of the 
alternative modes employs only the DEBE-333. 



Community 

The Upper Perkiomen School District is comprised of four small 
boroughs and three townships, with a total population of 19,000* The 53- 
square mile district covers the northern end of Montgomery County and an 
adjoining township in Berks County* It is a middle class, rural, conser- 
vative community with many residents commuting to the Lehigh Valley and 
Philadelphia areas for employment. 
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Program Initiation , PXannlng^» and Sr.art-up^ 

Initiation 

The Utility-Based Professional Staff Development (UPSD) program was 
initiated in 1977 by the superintendent, after he heard a presentation by 
Jerry Melton, who was at that time assistant sjjperintendent of the Newport 
News School District in Virginia, The teacher supervision approach that 
Melton described appealed intuitively to the superintendent. 

Planning and Start-up 

The process of adopting the program included extensive contact with 
the Newport News School District and involvement of Upper Perkiomen staff 
members 3.nd others, 

• The superintendent and assistant superintendent visited Newport 
News. Then, three groups — each including teachers, administrators, 
and board members — went to Newport News for three days each. Upon 
return, each group recommended adoption of the program, 

• A Newport News administrator (Bill Ethridge) was invited to Upper 
Perkiomen to conduct a week- long "theory and indoctrination'' 
session with 12 teachers. They also recommended adoption, 

• The approach was presented to the total staff during an Inservice 
day* At the direction of the superintendent, a vote was held at 
the end of that inservice day. Approximately 55% of the staff 
voted in favor of adoption. (Approximately half of the remaining 
staff members voted against adoption and half were uncertain.) 

• The teachers' association was approached during the above meeting 
but decided not to become involved formally in the decision. Their 
position was that the program was an educational, not a union, 
function. 

• Community support was also enlisted. Organizations such as the 
Lions and Rotary Club favored adoption. An ad-hoc Taxpayers' Union 
in existence, at the time, was opposed because of the finances the 
program would require. 

During the developmental phase of the program, various resources were 
used. The superintendent and UPSD trainer, a high school reading specialist 
who had been recommended for the training position by a group of 12 teachers, 
spent one week on the West Coast with Madeline Hunter and Ernie Stachowski 
and one week in Newport News. They invited three people from the Newport 
News Department of Staff Development to spend a total of approximately 23 
days in the Upper Perkiomen district training and/or advising staff. They 
also used materials such as writings by Thomas McGreal (1983), and Joyce 
and Showers (1980); the effective schools research; BTES reports; and Bloom's 
taxonomy in conceptualizing the program, UPSD evolved out of these various 
resources, 
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Program Funding 



Annual Costs 

The approximate costs per year since the program's inception have 

been: 



A llocation of Monies 

The largest expense has been the salaries (and fringe benefits) of 
teachers during training. During the adoption phase, expenses included 
trips to Newport News and hiring a consultant to conduct a week-long work- 
shop. During program development, expenses included sending the superin- 
tendent and program trainer to California and Newport News for one week 
each, bringing in Newport News staff for approximately 23 days, the 
trainer's salary, and purchasing materials. When teachers were trained in 
the process, money was required for the salaries of teachers,* substitute 
teachers, two trainers and materials. Current operating expenses Include 
training new teachers in UPSD. The district has also brought in consul- 
tants, participated in two study councils, and sent teachers for additional 
training in staff development related initiatives other than UPSD, since 
1982. 

Source of Monies 

The district initially obtained a three-year federal grant (Title 
IV-C) of approximately $100,000 to support the implementation of the pro- 
gram. Other sources of money included allocations from the school board 
and staff development funds from PDE. 



Administrators; Content and Process 

Administrators have received extensive training. They visited Newport 
News, and later attended a one-to-two week training session that Bill 
Ethridge from Newport News conducted in Upper Perkiomen. They attended a 
four-day workshop led by the director of Pittsburgh's teacher supervision 



*The superintendent included both the salaries of teachers and the 
salaries of substitute teachers used to cover the regular teachers' classes 
when they were out of class for UPSD training, in the cost of training. 
This was an arbitrary decision and from one perspective, presents an 
inflated projection of the cost of training. 



UPSD 
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78- 79 

79- 80 

80- 81 

81- 82 

82- 83 
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Staff Development: Content, Process and Evaluation 
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program. They met with various consultants. The training focused on the 
content of the Hunter instructional model and practice in various clinical 
supervision procedures ♦ 

Teachers: Content and Process 

Beginning in 1979-80 » teachers were trained in the UPSD approach* 
Training topics, in the order of their presentation, were teaching to an 
objective. Bloom's taxonomy, set, closure, task analysis, covert and overt 
behavior, monitoring the learner and making adjustments, motivation and 
reinforcement, retention and transfer, and Instructional skills. 

Teachers were trained in small groups of 12-15. Each training cycle 
lasted 6-8 weeks and consisted of one day of group instruction with the 
trainer followed by five days of practice and observation while teaching.* 
The observations were conducted by the trainer. 

Eighty percent of the teachers were trained during the first three 
years. Each training group included a cross-section of teachers from dif- 
ferent grade levels. Some follow-up training has been provided during in- 
service sessions conducted by the district and the lU. 

Initially, training groups consisted of volunteers. When they had 
been trained, administrators persuaded others to volunteer. New employees 
were required to participate. Several hold-^outs remain. 

Evaluation of Staff Davelopment 

Two members of the college of education at Pennsylvania State 
University evaluated the UFSD in 1981. They found that the istaff was very 
supportive of the program and reported that it gave them an increased sense 
of competence and professionalism* They raised questions about the pro-* 
gram's integration into the district's ongoing supervision and improvement 
programs and about the resistance of some staff to it. 

Thfe staff development trainer also asked members of each training 
group to evaluate the training. The results were not compiled across 
groups, but the data indicate that participants thought they understood the 
content presented (although some needed additional Input or practice) and 
that, overall, staff reactions to the program were positive. 



^Initially, there were six days of practice and observation; that was 
reduced to five days. 
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Staff Developnient Schedule /Timeline 



School Year 



Approximate 
Number Trained 



(1) 
(2) 
(3) 
(4) 
(5) 
(6) 



1979- 80 

1980- 81 

1981- 82 

1982- 83 

1983- 84 

1984- 85 



40 
30 
20 



20-24 



10-11 
10 



Implement at Ion of District Supervision/Evaluation System 



Ph i lo s o phy / Po 1 icy 

The district's philosophy is that teachers' Instructional skills are 
critical and that all teachers employed by the district are satisfactory. 
It Is believed > however, that supervision through the UPSD program can help 
improve the skills of all teachers, even those who are above average. The 
UPSD and the various supervision options are viewed as an ongoing process 
of supervision/evaluation. Some quotes from key administrators Included: 

• "We (observers) are here to catch you doing something right... 
We're going to build on your strengths/' 

• "The backbone of any quality educational program is the 
Instructional skill of the classroom teacher." 

• "We assume that all of these people are cocT^otent teachers. They 
are good teachers and they want to become better teachers." 

Requirements 

The district has five alternative modes of supervision/evaluation. 
Supervisors and activities vary according to mode. Each teachar selects a 
supervision/evaluation mode annually. The modes arei. 

• Principal-Teacher . The principal is the clinical supervisor, (In 
the high school, this responsibility is shared with the two 
assistant principals.) 

• Coilegial Mode . Teachers observe each other. Informal criteria 
for inclusion are five years of experience and an absence of 
questions about competence. 

• Reading Supervisor and Teacher . The reading supervisor functions 
as the clinical supervisor. This mode is open only to elementary 
and middle school teachers. 
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• Staff Development Coordinator-Teacher . The staff development 
coordinator serves as the clinical supervisor.* 

• Evaluative Mode . This mode Is viewed as "strictly evaluative." The 
DEBE-333 is used as the basic instrument. Non-tenured teachers are 
required to select this mode. Other teachers may also choose this 
mode of supervision/evaluation if they desire* 

The number of observations required per year varies according tc mode, 
but is not less than one. In all but the evaluative mode, a clinical 
supeirvision approach is followed — pre-conference , observation, write-up, 
and post-conference — but the exact procedures used vary* Pre-conferences 
are not always held* Some observations are scheduled in advance and some 
are not* Formal reports may be prepared before or after post-conferences, 
or not at all* Some principals simply complete a form, some nothing. Some 
do not hold conferences or write reports until after at least three obser- 
vations* Reports are placed in teachers' files, but only the reports of 
non-tenured teachers are sent to the district office* This process gives 
the principals maximum flexibility to use the program for instructional 
improvement. 

Organization and Process 

The district supervision and evaluation process Is carried out by 
the principals, assistant principals, reading supervisor, staff development 
coordinator, and supervisor of curriculum, instruction, and personnel* 'ihe 
latter observes non-tenured teachers and helps oversee collegia 1 super- 
vision. The superintendent and assistant superintendent manage the process 
and observe teachers. Principals are expected to spend a minimun. of one- 
third of their time on supervision; the superintendent spends 30%-35% of 
his time on it. 

Principals are responsible for supervising/evaluating 27-31 teachers 
and specialists each* (The high school principal is directly responsible 
for 29 staff members and shares responsibility for the remaining 29 with 
the two assistant principals). With the alternative supervision/evaluation 
modes > principals' responsibilities vary from clinically supervising 
teachers, to observing them and completing DEBE-333 forms, to working with 
teachers who are using the collegial supervision mode. 

System Monitoring 

Several approaches have been used to monitor the UPSD program* 

• Initially, principals were required to put everything in writing 
and submit it to the district office. 



*This mode was offered during the first four years of the program* 
The district staff developer is now a full timsj principal. In place of 
this mode, teachers may now choose the district writing program specialist 
as their clinical supervisor. 
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• The superintendonc , assl^Jtant superintendent, and staff development 
trainer sometimes partici;jate in observations and conferences. 

• Observations and conferences have been videotaped and then viewed 
by administrators* 

• Supervision/evaluation responsibilities are included in 
administrators^ "MBOs" (Management By Objective) system. 

• The supervisor of curriculum, instruction, and personnel monitors 
collegial supervision. 

Generally, central office administrators are satisfied that the process has 
been implemented effectively and has been standardised across schools. 



The high school principal (9-12), two middle school principals (re- 
sponsible for grades 5-6 and 7-8, respectively) and two elementary school 
principals (K-4) were interviewed for their perceptions of the district's 
teacher supervision/evaluatioa system* 

Relationship Between Teacher Supervision and Teacher Evaluation 

Administrators said that the district distinguishes between teacher 
supervision and teacher evaluation* They described supervision as on- 
going, formative » and geared towaid improving instruction; they described 
evaluation as more suiranative and negative. Some acknowledged, howex^er, 
that it is difficult to separate them. One said, "You're never going to 
separate the two." Another stated, "I don't thirk yon can separate the 
two. • , ^Teachers are going to see any administrative observations as super- 
vision," The latter said, however, that the collegial mode "clearly sepa- 
rates them.'' 

Utility of UPSD and Clinical Supervision Training 

Generally, administrators considered their training extensive and 
effective. They thought almost all of it was useful; the only exceptions 
were a few consultants who were not as sophisticated as Upper Perkiomen in 
teacher supervision/evaluation. All but one expressed a need for additional 
training — in conferencing, note taking, peer observation, goal setting, 
career ladders, dealing with experienced teachers, and in the job categories 
(other than instructional skills) which comprise Hunter's description of 
teaching. 

Feasibility of Implementation 

Generally, administrators seemed to consider that the process of 
implementing the program was smooth. Most reported spending 30%-40% of 
their time on teacher supervision; however, several (3-4) still felt that 
lack of time was a problem. The district attempted to deal with this 
problem during the program's early stages when the superintendent told 
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principals that if they did not decide how to free some time for the 
process he would make the decision for them* One principal said he now 
delegates more paperwork to a secretary. Another factor that encourages 
principals to spend time on the process is that the MBOs, of some, state 
thst they will devote at least one-third of their time to supervision. 

One principal said that the fact that the program was voluntary 
initially divided staff into two groups according to whether they supported 
or opposed it» Later, the program became less voluntary* That increased 
staff mistrust of administrators. 

Two principals described additional implementation problems. One felt 
some discomfort with the collegial mode of supervision. He felt that per-- 
haps administrators should become more involved in it in order to continue 
the program's advantage of increasing communication between teachers and 
administrators, and monitor the process. The other administrator said that 
teachers did not volunteer for the collegial mode as enthusiastically as 
expected, requiring the district to mandate it for some teachers. 

Impact of Supervision 

All administrators said they thought that the supervision program, as 
currently implemented, plays an important role in helping teachers improve 
their performance. Several said that it improves communication among 
teachers and between teachers and administrators. 

• "It has given people a common language." 

• "Doors are beginning to open, walls are coming down." 

• "It focuses teacher discussion more on instructional skills and 
less on social matters." 

• "It got administrators out of their offices, which automatically 
improved instruction." 

Other impacts described by administrators included establishing expec- 
tations, and helping teachers improve lesson planning, statement of objec- 
tives, and instruction. One administrator said that the system has helped 
principals identify needs for improvement and assisted teachers in streng- 
thening their skills. Another said that it has improved classroom manage- 
ment « 

Impact of Evaluation 

Administrators who were asked about the impact of the DEBE-333 (and 
some were not) found it difficult to identify effects. One- said that It 
gives people practice signing their names. 

Perceived Strengths 

Administrators described many strengths of the UPSD program. They 
said that the existence of alternative supervision /evaluation modes encou- 
rages more people to become Involved, and that teachers may extend more 
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choices to students subsequently. Another strength is the absence of a 
requirement for written reports following each observation; this enables 
administrators to convince teachers that the system^ s purpose is to super- 
vise rather than evaluate. The program also changes the administrative 
role, making it less threatening and more helpful. Other strengths that 
principals named included the amount of observation that principals are 
required to conduct, the initial voluntary nature of the program, the use 
of a good format that permits considerable flexibility, and the lesson 
analysis form. Administrators also reported strengths that many teacher 
supervision/evaluation systems share — the development of a common language 
and establishment of clear expectations. 

Perceived Problems and Needs 

Administrators identified relatively few problems or needs. Their 
perceptions of their own training needs were described earlier. Several 
expressed a need for additional time to conduct the process. One principal 
said that the district needs to devise a way to evaluate the quality of 
supervision. Another said that some concepts used in the system should be 
defined more clearly. 



T eachers' Perceptions 

Thirty teachers were interviewed regarding their perceptions of 
supervision and evaluation in the Upper Perkiomen School District, Eleven 
taught in elementary schools, six in the middle school, and thirteen in the 
high school. 

Relationship Between Supervision and Evaluation 

Some teachers were able to distinguish between supervision and eval- 
uation. Several described the latter as "rating" and said that the former 
was "not rating," One said that during supervision the principal "is not 
trying to evaluate me, he's trying to help me improve," Another Jiade a 
very similar statement. Some, particularly secondary school teachers, dis- 
agreed. Some said there is a "fine line" between the two or that they "fit 
together but can be apart also," Others said they are definitely related. 
They said that program terminology is used In evaluation regardless of 
whether or not teachers have been trained or that each supervision mode is 
evaluative. Also, teachers appeared to view supervision as evaluative 
because: (1) results are put in writing, and (2) they are put into 
teachers* files. Another problem they cited was that the program has 
become mandatory for new teachers. 

Satisfaction With Input 

Most of the teachers who were interviewed said that they, personally, 
did not have any input into the system. Some said, however, that otlier 
teachers did have input. Some said that teachers had little input. They 
said that the district decided to adopt the approach even before sending 
the initial groups of teachers (who purportedly played an important role in 
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that decision) to Newport News » that the district asked teachers if they 
liked the program — as if they had a choice--but later said they had to do 
it anyway, and "I felt like it was being pushed down my throat," 

Utility/Impact of UPSD and Supervision 

Many teachers reported that the UPSD training and supervision were 
useful and had an impact on their teaching, although many others attempted 
to minimize the program's impact. At least eight teachers said that the 
program has influenced their instruction. They said, for example » that it 
has affected their planning, questioning behaviors, use of the chalkboard 
and other media, student involvement » and checking for understandings They 
said, "I've learned to involve xay students more^ having them give me more 
feedback instead of me talking to them." "I've changed the lesson structure 
and the questioning." Some teachers reported that the training and 
supervision improved their understanding of how and why their own teaching 
behaviors affect student performance. 

Many interviewees said that the system increased their awareness, gave 
them a common vocabulary, or provided new terminology for already-familiar 
concepts. Some viewed this as helpful, saying that they had been in a rut 
or that it helped keep them on their toes* Some questioned whether these 
effects were sufficient to Justify the time and money spent on the program. 
A few were quite cynical and minimized the system's impact on them« 

Several teachers reported impacts from the collegial mode of super- 
vision* Interestingly, some of those teachers named benefits derived from 
observing others rather than from having their own teaching observed. They 
liked to see what other teachers were doing and to obtain ideas from them* 

Some teachers were asked to estimate the proportion of their teaching 
skills /knowledge that arose out of three sources: (1) college and 
university education, (2) experience, and (3) UPSD. The proportion attri- 
buted to UPSD ranged from 10% to 40%; the median was 2S%* 

Utility of Evaluation 

Most inter\fiews focused on the supervision system. Due primarily to 
time limitations, the evaluation system usually was discussed only as it 
related to the supervision system* Teachers who did mention the DEBE-333 
appeared to view it as a formality* As one teacher said, "It is always 
trotted out* . .annually. ** so that we can sign it." 

Fairness of Evaluation 

Similarly, very few teachers talked about the fairness of the. 
DEBE-333. One did say that it has been abused by administrators who have 
exerted pressure on some teachers by observing them frequently. The 
teacher considered that unfair. 
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Strengths 



When interviewees were asked to name the major strefigths of the super- 
vision system, they mentioned characteristics of the program design and of 
program implementation as well as program effects. All three will be des- 
cribed here — although the effects were similar to those described earlier 
in the section on program impact. 

One program design strength was the existence of alternative super- 
vision modes. Also, the availability of the collegial mode "makes you feel 
more responsible" and "allows teachers to talk to one another about instruc- 
tion." Another program design feature that teachers mentioned was the op- 
portunity for practice and the immediate feedback during training* Related 
to thatt some teachers liked the conferencing because it was a vehicle 
through which they could receive feedback and explain events that occurred 
during observations. Another design characteristic that was named as a 
strength was that observers do not assign numerical ratings. These cha- 
racteristics seemed to reduce the threat of the supervision system, "You 
don't feel like they are out to get blood." 

One implementation quality that teachers mentioned was that the staff 
development trainer was not threatening. . Teachers also responded favorably 
to administrators' assurances that the system was not lock step; they said 
that teaching is an art, and creativity and individuality cannot be denied. 
Another implementation quality mentioned was the district's allocation of 
time for the process. 

The system's strengths named by teachers that referred to program 
effects were that it has given teachers a common language, an awareness of 
what they are doing, and knowledge of the criteria used during obser- 
vations. Also, it has helped improve instruction. 

Needs/Problems 

The major need that teachers described was for UPSD follow-up and 
review. One teacher said that there should have been more post-training. 
Anothc-r suggested that an annual full«»day (or even half-day) "refresher 
course" should be held to at least allow teachers to aiscuss their exper- 
iences. One recommended that the staff development trainer conduct a 
review session. Other needs included visiting other programs, observing 
other teachers, focusing on other areas of Madeline Hunter's teacher job 
description model, and resuming the videotaping of collegial mode partic- 
ipants. 

Teachers described two problems, both of which referred to the process 
of implementing the system. First, the amount of out-of-class time re- 
quired tor training created problems for some teachers. They did not like 
having to leave their classes with substitutes so frequently. Some substi- 
tutes had difficulty following teachers' plans satisfactorily. Sometimes 
substitutes were not available and students were sent to study halls or 
classes were covered by other teachers. It was suggested that training 
should be conducted during inservice days rather than during school time. 
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The second problem was that adminiatrators ' initial messages about the pro- 
gram and their later actions were not consistent, creating resentment and 
mistrust. According to teacners, administrators said initially that the 
program was voluntary and non-^evaluative. T.ater> however > some teachers 
felt that they had little choice; administrators expected them to partic- 
ipate* Training was mandatory for new teachers. Some teachers believed 
that the program was definitely evaluative. They viewed the process of 
observing and conferencing as evaluative* or commented that program termi- 
nology was used on DEBE-333 reports* 

District Administrative Staff Overall Reflections 
On Program Design, Operation and Impact 

Perceived Strengths 

When asked about program strengths » most administrators described 
Impact rather than design and implementation* (When they described program 
design ai?d implementation > their tone and non-verbal behaviors often con- 
veyed a sense of satisfaction, but they did not verbalize those feelings*) 
One interviewee did describe design and operation strengths: extensive 
teacher involvement and the availability of alternative supervision/eval- 
uation modes* 

The program impacts that administrators described Included the 
following. 

• Common language and expections ("ground rules") have been 
established. 

• Staff are more knowledgeable about teaching effectiveness and more 
able to recognize it* 

• Principals are more effective supervisors. 

• Teachers are more effective instructors* 

• Teachers have become more professional* They are more aware of 
their skills and have higher self-esteem, They feel more ownership 
for the schools. They focus more on instructional methods and are 
more aware of the need for planning. They are more open and 
willing to share with one another* Their attitudes toward staff 
development have improved* 

Problems 

Administrators described the following problems* 

• Creating readiness for the program. They had to build trust with 
teachers to help them overcome suspicions about the system* Also, 
they had to convince principals of the program's value. This was 
administrators^ major problem, and they still feel uncertain about 
how to create readiness* 
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• Developing teachers' skills to a level where they can "go beyond 
the template." 

• Maintaining the system. 

• Obtaining funds for development training. This was viewed as a 
problem "but not that difficult," 

• Finding time. 

Changes Made or Things They Might Do Differently 

Several rather substantial program modifications have occurred, 

• The alternative modes of supervision we re^ added. 

• The content of staff development has been expanded. For example, 
it now includes learning and teaching styles, higher order 
thinking, and writing instruction* 

• The training program was compressed in time. 

• Principals are more involved In the supervision process. The staff 
development trainer has moved to a new position as a district prin- 
cipal. 

Administrators also described several things they would do differ- 
ently. 

• Present the program to teachers as mandatory. 

• Tell teachers that the program is permanent. Some teachers 
expected to be finished with it after completing the training. 
Later, they asked why they were going through it again* 

• Conduct a more formal evaluation of the program. Review the 
effectiveness of the various supervision modes. 

• Involve more teachers during the early stages. 

• Involve principals to a greater extent. Have them do more 
teaching. This would increase their credibility. 

Assistance They Could Have Used 

Administrators wish that more leadership had been available from 
colleges and universities and from state-level organizations. They noted 
that PDE might have provided academies for administrators on the roles of 
superintendents and principals in instruction and supervision. They felt 
PDE should provide training of trainers. PDE should maintain contact with 
people who are experts in TS/E on an ongoing basis and share new infor- 
mation with school district staff. 
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Advice to Others 



Administrators offered the following advice to other districts that 
are considering adopting similar programs • 

• The superintendent and board must make a cotomitment to the program. 
This might include signing a five-year employment contract for the 
superintendent . 

• Administrators must understand the program concepts thoroughly 
themselves. They should be trained before teachers. 

• The district will have to go through a readiness stage with 
teachers. Readiness might be facilitated by distributing written 
informationt attending conferences > and making site visits. 
Teachers should be informed of the program rationale. They should 
be told that the program will not be used to rate them. 

• Teachers must be given sufficient training, 

Supmiary of Findings and Discussion of Implications/Issues 

Summary of Findings 

Upper Perkiomen's teacher supervision system has several majpr 
strengths. First, the system was carefully planned and implemented. Many 
administrators, teachers, and others were introduced to the system and 
given an opportunity to have input into the adoption decision. Many 
scholars and educators who had experience with similar systems were con- 
sulted. Extensive training was provided for teachers. Many resources Were 
devoted to the activities. 

Second, the system is flexible and provides many options to teachers 
and administrators. The alternative modes of supervision, the flexibility 
in preparing observation reports, and the absence of a requirement that 
reports be sent to the district office particularly strengthen the system. 
Principals and teachers are treated as professionals who can be trusted. 
Threat is reduced, 

A third strength is that a capable and trusted insider was used as the 
primary trainer of teachers. He was selected for that position by a group 
of teachers. Teachers reported that they respected and trusted him. 

The UPSD has relatively few weaknesses cr problems, although teachers 
and administrators acknowledged several. A major problem for some teachers 
was that administrators' initial statements and later actions were incon- 
sistent, creating some resentment and mistrust. Administrators told 
teachers that the program was voluntary* Later, however, they coaxed some 
teachers into participating; and they mandated it for new employees. 
Similarly, administrators told teachers that the system was non-evaluative 
but later used its terminology in teachers' evaluation reports. Other pro- 
blems included the amount of out-of -class time required for training and 
insufficient follow-up or refresher trainingc 
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Implications /Issues 

Upper Perkiomen's teacher supervision system may provide many useful 
hints for districts that plan to adopt similar systems. Qualities that 
others might find particularly useful include the following. 

• Involvement of principals, teachers, and community members in the 
adoption decision* 

• Alternative modes of supervision and extensive staff training. 

• Use of an internal trainer who is respected/trusted by teachers* 

• Flexibility in the process of writing up observations. 

• The absence of a requirement that observation reports be sent to 
the district office. 
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